Controversially the final salary pension, in my opinion, is one of the main issues. As this benefit is so disgustingly good that people generally do not leave unless made redundant or in a box. The result is that the opportunity to progress is limited, compounded with when positions do arise the amount your salary can increase is capped at 10% (unless a Head of Department or Director which happens every ten years maybe) even though they advertise roles to external candidates at much higher salaries!
In addition to this the appraisal system is a joke. If you perform unbelievably well, reinvent the wheel (and trust me, most managers need this in order to get the top grade), due to government restrictions, the maximum pay rise you could earn per annum was 1.3%. If you sat around doing nothing it is 0.7%. This isn't going to change until the next general election. So not only after years of service you will find yourself 15% behind inflation, which when living in London makes you worse off over time, but there is a culture of apathy where good performance isn't rewarded and poor performance isn't penalised. The result is senior managers who are just coasting waiting for retirement as it is too expensive to remove them, despite failing within their role. There is a silo culture and departments do not talk to each other constructively as barriers have been formed in order to justify some peoples positions and jobs! The result is all the talented & passionate staff leave after 2-3 years as there is no reason to stay unless you are happy coasting in your role (if able to secure one of those). If you can play the internal politics then you can do reasonably well here (although not very well paid).
Also VisitBritain is a tourism marketing agency. However the government treats the organisation like other government departments and therefore receives an annual budget. The problem being that all the money needs to spent AND activity to be completed AND evaluated by the end of the financial year. Which makes marketing campaigns limited and unattractive to trade and commercial partners who struggle to justify the ROI with such a short term-ism attitude. The whims of the politicians (who know nothing of Tourism) are a constant frustration. Prepared to drop everything in an attempt to do something in order to make a minister look good.