Agency head of environmental protection external programmes Martin Brocklehurst told NCE's Waste Summit that the Agency was going to come down heavily against "sham recovery" of recycled aggregate.
Brocklehurst said the Agency was preparing two prosecutions against rogue contractors. They had been mixing materials such as asbestos into recycled aggregate products.
"We are finding sham recovery to be a big problem at the moment," said Brocklehurst, The quality of recycled aggregate is strictly regulated under the Environment Agency's Aggregate Protocol but industry experts told NCE the protocol had not been widely enforced by the Agency.
The drive was welcomed by Tarmac recycling director Alan Sheppard.
"The Agency has said that it wants to start taking some action on this issue and we would endorse that," he said.
"We don't know what the scale of the problem is but I think it is potentially a big problem because people are not carrying out the full testing and screening required.
"As a result of that, waste is sometimes being sold as recycled aggregate and that is giving the product a bad name."
Recycling experts also urged the Agency to crack down on contractors mixing "inert waste" with plasterboard and gypsum so they can landfill it cheaply.