One thing you have to decide is: do you want to know that scratches have occurred after you have used a scraper on tempered glass? If you have a good waiver signed it will not be on you. Some would rather not scrape and risk seeing scratches no matter what. It's your take on things, really. There is the option of using alternative methods to a proper scraper, but they are almost always more labor intensive, time consuming, and costly as far as materials. You can also give the owner the option of allowing you to do the best you reasonably can without spending an eternity. Every option has its pros and cons.
The unfortunate thing is that as long as you are doing your due diligence it is always someone else's fault that you and the building owner have to deal with this; whether the glass manufacturers who should be doing cleaner tempering, the clowns who left junk on the glass and walked away, or the contractor who should have made sure this didn't happen on site.