I recently came across a Crawler Error:
404-HTTPNOTFOUND for some sharepoint sites, that were actually reached by browser. I gave an Index Reset in my Search Application; for ensure that data remains in sync I used my Microsoft.FASTSearch.Powershell snapin for clear the content collection
“Clear-FASTSearchContentCollection -name sp”
Then a full crawl resolve my issue!!
No More 404..