The Office of Tax Simplification (OTS) has called for reform to the tax system for pension fund withdrawals, which it says is not widely understood by most people.
The report raised particular concerns with the use of emergency tax codes for lump sum withdrawals from personal pensions.
In the majority of cases, over-55s making their first withdrawal from a personal pension do not have access to an up-to-date tax code, so they are usually taxed at an emergency rate.
This often results in overpaid tax, which the taxpayer then has to claim back.
According to Royal London, savers have had to reclaim more than £300 million in overpaid tax on pension withdrawals since pension freedoms were introduced in 2015.
Steve Webb, director of policy at Royal London, has criticised the system, claiming it is "run for the convenience of HMRC, not the taxpayer".
The OTS is planning to work with HMRC to identify other options, and to improve guidance around the topic.
"It is very welcome that an official body like the OTS has recognised the current way pension withdrawals are taxed is baffling.
"The sooner HMRC work with others to come up with a fairer and simpler system, the better."
Speak to us about accessing your pension.