Hey folks, I’m trying to figure out the best way to use Solr in my e-commerce project. I’ve got a SQL database, but I want to use Solr for my category pages to show product info like stock and price. I’m also looking to hide out-of-stock items.
I’m using Delta import queries to speed things up, but I’m worried about page performance and data accuracy. My database queries are pretty complex with lots of joins, so I’ve had to create multiple entities in Solr. This makes data uploads slow, even with delta imports, so I’m not getting real-time stock info.
I’m stuck between two options:
Put all my data in Solr and get everything from there (worried about performance and real-time data)
Just get product IDs from Solr and fetch other details from SQL (not sure if this will be fast enough)
Any advice on the best approach? What’s worked well for you? Thanks in advance for any tips!
I’ve faced similar challenges with Solr in e-commerce. From my experience, a hybrid approach tends to work best. Keep core product data in SQL, but utilize Solr for search, filtering, and faceting. This leverages Solr’s strength in these areas while maintaining data integrity in SQL.
For real-time updates, consider implementing a lightweight message queue system to push critical changes (like stock levels) to Solr immediately. This can significantly improve data accuracy without overburdening your system.
Regarding performance, I’ve found that denormalizing data in Solr can help. Create a single, flattened document per product with all necessary fields. This reduces the need for complex joins and can improve query speed.
Lastly, implement smart caching strategies for frequently accessed data. This can dramatically reduce load on both Solr and SQL, improving overall system performance.
Hey Liam_Stardust, interesting dilemma you’ve got there! I’ve dabbled with Solr in e-commerce before, and it can be tricky to get the balance right.
Have you considered a hybrid approach? You could keep your main product data in SQL, but use Solr for search and filtering. This way, you’re playing to the strengths of both systems.
For real-time stock info, maybe look into a message queue system? That could help you push updates to Solr more efficiently without slowing down your main database operations.
I’m curious, how big is your product catalog? And what’s your current update frequency? Those factors can really influence the best strategy.
Also, have you thought about caching frequently accessed data? That could help with performance without putting too much strain on either system.
What’s your current response time for category pages? It’d be interesting to know what you’re working with and what your target is.
Keep us posted on what you decide to do! Always fascinated to hear how others solve these kinds of challenges.
yo liam, solr can be a beast in ecommerce! i’d go with a mix - use solr for search n filtering, keep core stuff in sql. for real-time updates, maybe try a lightweight message queue? it can push critical changes to solr fast. also, flattening data in solr might help with speed. good luck mate!