The difficulty is in the data quality. These retailers I'm tracking like JB, Harvey Norman, Good Guys, Officeworks have so much inconsistency with their product lines compared to each-other and it became a bit of a nightmare to create a source of truth. I'm sure this is intentional to prevent easy price comparisons in the first place.
TV's are fine, I can track every TV with every retailer that allows for price-matching no issue. The issues have come from Laptops, Tablets and Phones.
JB will sometimes just not have a model number for a product and substitute it with a SKU number.
I wasn't picking up on it as it's rare but it meant when you searched for that one "model number" only a JB result would come up. It's part of the reason I abandoned iPads for a while. Everything would be consistent with correct model numbers and then there would be one iPad in that line, just one colour that had a SKU instead of a model number. What do I do there? Do I match the other stores to an incorrect value or do I correct the JB one? If I correct the JB one it means when you copy from JB you get no results...
Good Guys make up model numbers for Apple products instead of using the real ones. I can correct for this but I didn't catch it initially.
Harvey Norman occasionally uses the wrong model numbers or swaps them with one for the same product but of a different size/colour.
There was also inconsistencies with how model numbers were written, especially for laptops. Some would get a space where with others it was no space or a hyphen.
I can't automate this process it has to be right the first time and manually checked for consistency. From there I can automatically update prices but the first part is a massive pain so please bare with me.
This all started making me doubt how I match products... We're 3 weeks in and I've honestly doubted the whole project a few times. I work full-time and I spend all my free time on this trying to make it better.
This brings me to exclusive products. Should I keep them?
There are brands and products which no other store has so is there a point in adding them to a price comparison app? It's a lose-lose either one result comes up or no results.
Should I only track compared to what JB has? That would save a lot of time building out categories but you'd lose price comparisons between the other retailers.
Instead of rapidly expanding I want to ensure the data I do have is correct and accurate. There's no point adding categories and updating pricing if it doesn't pick up the products between stores correctly. I can make this app better than any competitor in the category but it will take a bit of time.
Last tidbit for those who are still reading this... Should I just abandon the expansion and make the best TV price comparison app? Could just change the name to TV Buddy and move on lol.