2016-11-05 Saturday - Enterprise Search

I'm doing a bit of a refresh on the latest Enterprise Search options that are out there - and will use this blog post as my placeholder for my notes. Hopefully this will become a useful resource for others too.

Initially, I intend to examine the following (but may add others as time permits):

• Apache Solr (Latest v6.0 April 2016)
• http://lucene.apache.org/solr/ 
• http://lucene.apache.org/solr/resources.html#documentation
• http://lucene.apache.org/solr/quickstart.html
• https://issues.apache.org/jira/browse/SOLR/
• https://wiki.apache.org/solr/
• https://wiki.apache.org/solr/Join
• https://wiki.apache.org/solr/Solrj 
• https://wiki.apache.org/solr/UpdateXmlMessages
• https://wiki.apache.org/solr/DataImportHandler
• Use full-import command to do a full import from the database and add to Solr index
• Use delta-import command to do a delta import (get new inserts/updates) and add to Solr index  
• https://wiki.apache.org/solr/DataImportHandlerFaq

• https://cwiki.apache.org/confluence/display/solr/About+This+Guide
• https://cwiki.apache.org/confluence/display/solr/Major+Changes+from+Solr+5+to+Solr+6 
• https://cwiki.apache.org/confluence/display/solr/Introduction+to+Solr+Indexing
• https://cwiki.apache.org/confluence/display/solr/Indexing+and+Basic+Data+Operations 
• http://lucene.apache.org/solr/news.html#8-april-2016-apache-solr-600-available
• http://lucene.apache.org/solr/6_0_0/changes/Changes.html 
• https://www.openhub.net/p/solr
• https://en.wikipedia.org/wiki/Apache_Solr 
•  active community...plugins

• Elasticsearch (Latest: v5.0.0 Oct 26, 2016)
• https://www.elastic.co/products/elasticsearch
• https://www.elastic.co/guide/index.html  
• https://www.elastic.co/guide/en/elasticsearch/guide/master/index.html
• https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html
• https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started.html
• https://www.elastic.co/guide/en/elasticsearch/reference/current/general-recommendations.html 
• https://www.openhub.net/p/elasticsearch
• https://en.wikipedia.org/wiki/Elasticsearch
• "supports multitenancy"
• "rebalancing and routing done automatically"
• "percolating"
• "gateway...recovery of index after server crash" 

• Amazon CloudSearch
• https://aws.amazon.com/cloudsearch/
• https://aws.amazon.com/documentation/cloudsearch/ 
• https://aws.amazon.com/cloudsearch/pricing/ 

• Rocana
• https://www.rocana.com/ 
• https://www.rocana.com/rocana-ops/limitless-search

General Search Articles

• http://opensourceconnections.com/blog/2016/06/29/top-mistakes-made-in-search/
• https://findwise.com/Enterprise-Search-Findability-Report-2016
• http://sunset.usc.edu/classes/cs572_2015b/
• http://sunset.usc.edu/classes/cs572_2015b/CS572_HW_SOLR_WEAPONS.pdf

 Apache Solr Related Articles

• http://www.enterprisesearchblog.com/solr/
• http://www.solrtutorial.com/
• http://www.solrtutorial.com/basic-solr-concepts.html
• http://blog.kevindoran.co/solr-tutorial/
• https://www.quora.com/How-can-I-index-multiple-tables-in-Apache-Solr
• https://wiki.apache.org/solr/Join 
• http://www.toadworld.com/platforms/oracle/w/wiki/11121.collecting-indexing-and-searching-mysql-database-table-data-in-apache-solr
• https://gist.github.com/maxivak/3e3ee1fca32f3949f052
• After successfully adding a collection to Solr you can select it and run dataimport commands:
  
• full-import - use URL: http://localhost:8983/solr/myproducts/dataimport?command=full-import
• delta-import - use URL: http://localhost:8983/solr/myproducts/dataimport?command=delta-import
• http://blog.comperiosearch.com/blog/2014/08/28/indexing-database-using-solr/
• See: "Here we have carried out a full index of our database using the full-import command parameter. To only retrieve changes since the last import, we would use delta-import instead."
• See: "Finally, to schedule reindexing you can use a simple cronjob. This one, for example, will run everyday at 23:00 and retrieve all changes since the previous indexing operation"
• http://blog.comperiosearch.com/blog/2015/04/14/solr-indexing-index-sql-databases-made-easier-part-2/
• See: "Facets, also known as filters or navigators, allow a search user to refine and drill down through search results. Before we get started with them, we need to update our data import configuration. Replace the contents of our existing db-data-config.xml with" 

• http://solr.pl/en/2010/10/11/data-import-handler-%E2%80%93-how-to-import-data-from-sql-databases-part-1/
• http://solr.pl/en/2010/11/01/data-import-handler-%E2%80%93-how-to-import-data-from-sql-databases-part-2/
• http://solr.pl/en/2010/11/22/data-import-handler-%E2%80%93-how-to-import-data-from-sql-databases-part-3/  
• http://stackoverflow.com/questions/30837300/how-could-i-properly-configure-solr-to-index-my-oracle-database
• https://jasonrogena.github.io/2014/10/03/solr-with-mysql.html
• https://community.hortonworks.com/articles/52183/solr-indexing-the-database-tables.html 
• https://www.gadgetdaily.xyz/ssh-tunnelling-on-insecure-networks/
• https://coderanch.com/t/647210/User-Solr
• https://rahuldausa.wordpress.com/2014/05/16/real-time-search-on-40-billion-records-month-with-solr/
• https://www.lullabot.com/articles/finding-related-content-faster-with-apache-solr
• Indexing MySQL database with Apache Solr 
• https://www.youtube.com/watch?v=BuSCfi7AsLU 

Apache Lucene Related Articles

• http://lucene.apache.org/core/6_2_1/changes/Changes.html
• https://www.tutorialspoint.com/lucene/index.htm
• http://www.lucenetutorial.com/techniques/indexing-databases.html
• https://dzone.com/articles/lucene-database-oracle-five 

Elasticsearch Related Articles

• https://www.tutorialspoint.com/elasticsearch/index.htm
• https://www.tutorialspoint.com/elasticsearch/elasticsearch_basic_concepts.htm
• http://www.elasticsearchtutorial.com/ 
• http://www.elasticsearchtutorial.com/basic-elasticsearch-concepts.html
• http://opensourceconnections.com/blog/2016/09/09/better-recsys-elasticsearch/
• http://stackoverflow.com/questions/31755642/how-can-i-multiply-the-score-of-two-queries-together-in-elasticsearch 
• http://joelabrahamsson.com/elasticsearch-101/
• https://aphyr.com/posts/323-call-me-maybe-elasticsearch-1-5-0
• "You can lose documents if..."
• https://www.elastic.co/guide/en/elasticsearch/resiliency/current/index.html 

 

Comparison Articles

• http://cloudacademy.com/blog/elasticsearch-vs-cloudsearch/
• http://solr-vs-elasticsearch.com/ 
• http://logz.io/blog/solr-vs-elasticsearch/
• Solr: "Very well-documented"
• Solr: "Database integration"
• ES: "Lacks in documentation "
• ES: "Intuitive"
• ES: "If you need distributed indexing, then you need to choose Elasticsearch. Elasticsearch is the better option for cloud and distributed environments that need good scalability and performance"
•  Comparing SolR/Lucene/ElasticSearch vs. Cloud based Search Providers
• https://www.youtube.com/watch?v=JiMxmkMGrbU
• https://thinkbiganalytics.com/solr-vs-elastic-search/
• http://opensourceconnections.com/blog/2016/01/22/solr-vs-elasticsearch-relevance-part-two/ 
• [ES queries are verbose, but more readable... ]
• http://opensourceconnections.com/blog/2015/12/15/solr-vs-elasticsearch-relevance-part-one/
• "ES wins on matching" 
• https://www.datanami.com/2015/01/22/solr-elasticsearch-question/
• https://sematext.com/blog/2015/05/27/side-by-side-with-elasticsearch-and-solr-performance-and-scalability/
• "see more Elasticsearch clusters with issues..."
• "despite what you just read about Elasticsearch, we rely on it in our organization for several different products"
• "Elasticsearch is often for filtering and grouping – the analytical query workload – and not necessarily text search"
• "Elasticsearch is a better choice for applications that need to do not just text search, but also complex search-time aggregations"
• http://harish11g.blogspot.in/2015/07/amazon-cloudsearch-vs-elasticsearch-vs-Apache-Solr-comparison-report.html  
• http://www.flax.co.uk/blog/2015/12/02/elasticsearch-vs-solr-performance-round-2/ 
• https://sematext.com/blog/2015/01/30/solr-elasticsearch-comparison/
• http://db-engines.com/en/blog_post//55
• https://www.quora.com/What-is-the-difference-between-Apache-Solr-and-Lucene
• http://www.hcltech.com/blogs/elasticsearch-vs-solr
• Elastic: 
• "index documents of different index structure in a single Index"
• "automatic shard rebalancing"
• "supports complex nested types"
• Percolation Queries (i.e. for alert generation)
• Solr: 
• "have to simulate that inside your application or develop a custom search component."
• "complex distributed group"

• http://www.infoworld.com/article/3119213/analytics/beyond-solr-scale-search-across-years-of-event-data.html

Search Related Books

• https://www.manning.com/books/relevant-search
• https://www.amazon.com/Lucene-Action-Second-Covers-Apache/dp/1933988177 
• https://www.amazon.com/Solr-Action-Trey-Grainger/dp/1617291021/
• https://www.amazon.com/Elasticsearch-Action-Radu-Gheorghe/dp/1617291625/ 
• https://www.amazon.com/Elasticsearch-Definitive-Guide-Clinton-Gormley/dp/1449358543/
• https://www.amazon.com/ElasticSearch-Cookbook-Second-Alberto-Paro/dp/1783554835/ 
• https://www.amazon.com/Mastering-Elasticsearch-Second-Rafal-Kuc/dp/1783553790/
• https://www.amazon.com/Learning-ELK-Stack-Saurabh-Chhajed/dp/1785887157/