Not long ago a friend of mine approached me with this idea: build a software that would export LinkedIn search results to Excel. Apparently many recruiters and sales people are using LinkedIn for prospecting – finding potential buyers and candidates. They can search but there is no easy way to export results to CRM that they are using.
At first I was hesitant. I thought that this software would essentially fill a hole in LinkedIn. At any point of time, I thought, LinkedIn would implement this missing piece of functionality and we will be out of business.
However, as it turns out LinkedIn has no intention to add exporting functionality because they have products that fill this need: LinkedIn Recruiter and Sales Navigator. They are very expensive, by the way.
Also, there is a company that has exactly this and they are doing good. My friend talked to some recruiters and all of them told him that they use that product to export search result to Excel.
At about same time I’ve read ‘7 Day Startup’ book by Dan Norris. It’s a fantastic book, short and inspiring. The main idea is to create a product or service very quickly and launch it to see if people would buy it. So I got very excited about this. Competitor product was quite expensive – about $20 per month. I thought I could whip up a solution very quickly and charge less. If people start buying it we could decide how to ramp it up.
My idea was to create a Google Chrome extension that would scrape search results from LinkedIn website. Users would perform search as usual on LinkedIn, then click a small button in toolbar and get results in a text box. Then they would copy and paste results to Excel and from there they’d be able to do whatever they want with them.
What’s great is that Google has a special store for Chrome extensions. They even take care of charging money. People can buy an extension using Google Wallet.
Google also makes it easy to offer a trial experience. My idea was to let people to export just 20 results and then ask them to buy the full version if they want all results.
So feeling enthusiastic about this thing I have created my first extension in just one week. I published it on the Google Chrome Store and set the price to $19. The next step was to drive some traffic to the listing page.
I decided to make a YouTube video with the title “How to export LinkedIn search results to Excel”. People that search for that phrase would watch the video, find my extension and give it a try.
And then I remembered that I wanted to check LinkedIn user agreement to see if I can get away with scraping their website. The agreement couldn’t be more clear:
8.2. Don’ts. You agree that you will not:
…
Scrape or copy profiles and information of others through any means (including crawlers, browser plugins and add-ons, and any other technology or manual work);
…
Wow! I realized that by publishing this video I would essentially teach people how to do something that breaks LinkedIn user agreement. I just couldn’t bring myself to do it. Also, even though I wouldn’t violate the agreement myself (I just provide a tool) people would violate it by using my tool.
So I decided to back out of this project. It just didn’t feel right to me. I’m glad I didn’t spend long time working on it. Also, at least now I know how to create Google Chrome extensions.
Lesson learned: check website’s user agreement before creating a tool that scrapes it.
It was always a good idea to explore all the possibilities n legalities of things. There’s no failure, only lesson learnt.
Great read, Pavel! I was thinking of doing something similar for Gumtree – glad you mentioned the User agreement as they also have this restriction. Anyway, as they say in “The Lean Startup” by Robert Ries, pivot when you run into a road block and pivot often 🙂
Maybe other sites like realestate.com.au don’t have these restrictions and given property is so hot at the moment, there could be a market for this esp. in Australia? Just a thought 🙂
Also, another thought, given your expertise in this area, it would be great to see how you did this in a youtube video for example.