Utilising Solr in Drupal

It is know that the apache solr search module and all the dependency modules can handle nodes, entity types and taxonomies searching and indexing. However, the downside of this module is that it can not index non-Drupal content types or other data sources.

As a library/research institution and Government agency, our organisation has different data source end in SQL server which are managed by third party software to allow the curators and the volunteers to update it. How can we publish this data in such a way that it could be accessible/searchable to Web audiences and linkable to other services?

The answer for Drupal might be migration.  

If we do this, the challenges might be:

1. How can we manage/synchronise the updated data from different data sources system?  
   • The answer might be running a cron job automatically.
2. How can we manage the updated data structure from different data sources?  
   • For example, we created a content type for a particular data model to be shown in Drupal, then the business cases are changed with time?
3. What the performance will be if we try to import/migrate more than 2 millions nodes into Drupal, and make them searchable on website?
   •  We cannot answer this question because we don’t have the resources to test this solution.
4. How can we manage a large scale website with a small team.
   • There is no budget and resource.

In this case study, I will show you:

1.    Our unique solution to utilising Solr in Drupal for our organization;

2.    Some of our tool kits we built up based on Solr/Solarium, to manage our collection digital assets.