I am in a similar, but not identical, situation here in Canada. Unlike you, I no longer have British citizenship.
Your birth certificate will be British, like mine. You will need a Gender Recognition Certificate in order to change the gender on your birth certificate. The requirement for that is two years of RLE (real life experience) in your new gender. They announced an intention to change the regulations to remove that requirement, but that announcement was only a couple of weeks ago, so nothing is likely changed yet.
With your birth certificate changed, unless the Australian regulations are truly bizarre, you should be able to change the gender on all your other documentetion. That is the way it works here.
Your name change should be easier. Whatever the process is in your area, it is a local change where you live that matters. Here, for example, it is the province of Nova Scotia that changed my name. I just filled in the form, supplied the required documentation, paid my fee, and I received the certificate. All other jurisdictions, including the Brits, will accept my Nova Scotia name change certificate. Again, unless the regulations there are strange, it should be similar where you are.
You will have to research what the regulations are in the part of Australia where you live. Each jurisdiction handles name changes in its own way. Some use court orders, some use deed polls, some just require that you fill in a form.