Spiders, Extractors, and Crawlers are all software tools that go out and search metadata to retrieve results that can be helpful to your business. They are often used to search for emails. One popular tool, once endorsed by Mike Lipsey, president of The Lipsey Company, is the eGrabber, a lead generation software. I’ve used it for many years but customers have become so overwhelmed that I question the effectiveness of HTML mail any longer. So far, the tool I use that tracks emails comes standard in Salesforce and counts the number of times your email has been opened. This provides metrics that can refine your marketing program.

Spiders, Extractors, and Crawlers are helpful in other ways. I input search terms like Gardena manufacturing companies, California commercial investment properties, or new warehouse development, and while it works a lot like a Google search, crawlers search hundreds of search engines, not just those results that are contained on Google servers. The overwhelming problem you will face is a complete deluge of data that is almost completely impossible to parse unless you have a scientific way to manage all the results. I’ve found basic sorting and deleting can make a dent but not enough to be immediately useful.

Most non-technical people like me face two choices. Either buy tools that are specific and limited in their application or hire outside help with programing skills to extract more useful data.

Why would an SIOR want to bother with data extraction? Well, through data extraction I’ve found both tenants and deals I would not have found otherwise. In the world of high finance and hedge funds, data scientists pore over tweets, Facebook, and other data streams to acquire proprietary knowledge for trading purposes. Data management on this level is a major step to higher productivity.

I am not advocating hiring a staff of PhD’s to run your real estate analytics and build an empire. However, thinking about how you obtain data, process, and output it, will give you new insights and leads. My next step is to install Scrapy, a powerful and simple extractor. It’s written in Python, the language that runs kid’s toys like Arduino and Raspberry PI. But no, I’m not even suggesting learning that simple language. For a few hundred dollars, I found a Python coder on the internet and they will just give me a few basic commands I can use over and over to get the data I need. I’ve only skimmed the surface and am a novice myself, but data is a gateway to bigger and better deals.