That looks awfully similar to what I had happen with mine. User Mikezohsix also had a similar experience.
When did you buy the ipd cam?
I am running an IPD Turbo cam which i bought new from IPD around 5(?) yrs ago. It cam with the black coating. Ran for years on reused shims with in spec clearance for years, no issues. B230F+T using na manifold, JPipes, EVO8 turbo 15 psi.
Then i upgraded to 531 ported head with 40mm exh vlvs, 3" TB, KJet manifold & due to increased flow (but same exh manifold setup) EGTs went through the roof (1000 DegC+).
I now have better exh manifold & bigger turbo & egts are back to sensible levels.
However, since the EGT issue my exh vlv clearances keep closing up & when they reach 10 thou the cam eats a shim.
I stripped the head & found vlvs were 'cupping' (valve head sinking into the port), found when measuring as the underside of the vlv head is no longer flat, but sunk in the middle.
For now im putting this down to having softened the vlvs. The tempering temp for the grade of stainless used is around 800deg.
Im currently replacing the valves.
Point to note, is my #2 exh was the one which sinks quickest & eats the shim, same as your clearances show, coincidence??
Make what you will of my own findings & thoughts, but wanted to chip in and share the similar issues im having & working to resolve.