Larry it is almost comical but I have seen 7 figure transactions go south for spats of thousands of dollars. Usually it is because of underlying emotion rather then the economics of the situation. No matter what either party says, they always both fall back on the same line, “it is the principle of the situation” or something of that nature.
If this is a home you really do love and will be staying there for a long long time then between all parties and agents involved, somebody should just say “let’s get it done.” HLS really captured it right. Alot of people are making alot of money.
Now regarding earnest money, yes escrow is holding the deposit but as indicated by Rustico, escrow will need to receive authorization by both parties to cancel escrow if you cannot reach a resolution. Similarly escrow will also need to be authorized by the seller to release the entire deposit back to you. If there is a dispute (such as the seller keeping part of the deposit) then there are avenues you can take to try to retreive it. Consult with your realtor and they can discuss your options.
BTW it is certainly well within the sellers right to ask for a second opinion for inspections that you have had performed. Again, consult your realtor on this request. Perhaps having your inspector present while the sellers inspector performs their own inspection would be an amiable way to make sure everyone is on the same page?