This week MPs on the Environment, Transport and Regional Affairs Select Committee heard evidence from a group of lawyers and academics criticising the HSE's failure to properly investigate and prosecute after accidents leading to death and serious injury.
The facts support their accusations - there are many accidents and too few inspectors to carry out the necessary investigations.
Current HSE policy is focused on accident prevention. In construction as much as any other sector, all would agree that it is right that engendering a culture of safety awareness is much better than chasing culprits around with a big stick after accidents.
But lack of resources is a problem. The railway industry has been under the spotlight most recently following the Ladbroke Grove rail crash, which came at a time when the Rail Inspectorate admits to only having 50% of the inspectors it needs.
All over the country trains are running on track which is not yet fully checked or approved. Can this be right?
The point made this week was that there must be resources allocated specifically to enforcing the Health & Safety at Work Act.
It is a law, and when it is broken the offenders should expect prosecution. But the HSE simply lacks the resources to take on the often highly complex, long drawn out cases which so often result from serious accidents.
While we cannot simply lay blame at the HSE's door, there is much to said for having a special organisation to deal with the aftermath of accidents.
As an industry the construction industry has a massive role to play in helping itself become much safer. But if we are serious about tackling safety why not have a heavyweight, well resourced body to collect the evidence and secure the prosecutions in the higher courts?
Individuals running companies and setting management procedures must take responsibility for their actions and those of their employees.
If they do not perform as required by law then examples must be made of them.