Rebuild XDB indexing Issues
When dealing with SOLR and XConnect configuration in Azure, it's essential to consider all aspects before making changes. Based on a holistic view, it's clear that adjustments in the SOLR configuration for both Azure and XConnect Web App are necessary. This is in reference to the error "Solr xdb Rebuild not progressing from past 4 days," logged at this question.
The Required Configuration Changes:
XConnect Configuration Changes:
- One helpful resource for making XConnect configuration changes can be found in the article: Configuring the XConnect Search Indexer. This article serves as a comprehensive guide for addressing XConnect-specific changes.
- In making changes for SOLR, while the article How to Set the JVM Memory for SOLR Running on an Azure App Service provides valuable insights, it's important to note that other methods of making SOLR changes are also available, as demonstrated in the article: How to Increase SOLR JVM Memory.
- For SOLR changes, the following article How to Set the JVM Memory for SOLR Running on an Azure App Service can be a valuable resource for gaining insights into specific changes.
XConnect Specific Configuration:
- To make adjustments for XConnect, it's crucial to consider the environment-specific optimizations. Additionally, it's essential to carefully review and adjust the configurations for the XConnect files.
- SingleThreadedIndexerSettings in the sc.XConnect.SearchIndexer.xml file.
- IndexerSettings in the sc.Xdb.Collection.IndexerSettings.xml file
- IIndexRebuildFlow in the sc.Xdb.Collection.IndexWriter.SOLR.xml file.
- SolrWriterSettings in the sc.Xdb.Collection.IndexWriter.SOLR.xml file.
SingleThreadedIndexerSettings :
File path: C:\<Path to
xConnect>\App_data\jobs\continuous\IndexWorker\App_data\Config\Sitecore\SearchIndexer\sc.XConnect.SearchIndexer.xml
<SingleThreadedIndexerSettings>
<Type>Sitecore.Xdb.Collection.Indexing.SingleThreadedIndexerSettings,
Sitecore.Xdb.Collection</Type>
<LifeTime>Singleton</LifeTime>
<Options>
<Frequency>0.00:00:02.00</Frequency>
<DelayAfterError>0.00:00:05</DelayAfterError>
<DelayAfterRecurringError>0.00:00:30</DelayAfterRecurringError>
<RecurringErrorThreshold>1</RecurringErrorThreshold>
</Options>
</SingleThreadedIndexerSettings>
IndexerSettings :
File path: C:\<Path to
xConnect>\App_data\jobs\continuous\IndexWorker\App_data\Config\Sitecore\SearchIndexer\sc.Xdb.Collection.IndexerSettings.xml
<IndexerSettings>
<Type>Sitecore.Xdb.Collection.Indexing.IndexerSettings,
Sitecore.Xdb.Collection</Type>
<LifeTime>Singleton</LifeTime>
<Options>
<SplitRecordsThreshold>250</SplitRecordsThreshold>
<IndexPIISensitiveData>false</IndexPIISensitiveData>
<IndexAnonymousContactData>true</IndexAnonymousContactData>
<!-- In our situation we deliberatly make all data as anonymous in non production environment as per HIPA policy and PII policy -->
</Options>
</IndexerSettings>
IIndexRebuildFlow:
<IIndexRebuildFlow>
<Type>Sitecore.Xdb.Collection.Indexing.Rebuild.IndexRebuildFlow,
Sitecore.Xdb.Collection</Type>
<As>Sitecore.Xdb.Collection.Indexing.Rebuild.IIndexRebuildFlow,
Sitecore.Xdb.Collection</As>
<LifeTime>Singleton</LifeTime>
<Options>
<IncomingDataLagOnCompletion>0.00:00:05</IncomingDataLagOnCompletion>
<ParallelizationDegree>30</ParallelizationDegree>
<BatchSize>250</BatchSize>
</Options>
</IIndexRebuildFlow>
SolrWriterSettings :
File path: C:\<Path to xConnect>\App_data\jobs\continuous\IndexWorker\App_data\Config\Sitecore\SearchIndexer\sc.Xdb.Collection.IndexWriter.SOLR.xml
<Solr.SolrWriterSettings>
<Type>Sitecore.Xdb.Collection.Search.Solr.SolrWriterSettings,
Sitecore.Xdb.Collection.Search.Solr</Type>
<LifeTime>Singleton</LifeTime>
<Options>
<ConnectionStringName>solrCore</ConnectionStringName>
<RequireHttps>true</RequireHttps>
<MaximumUpdateBatchSize>250</MaximumUpdateBatchSize>
<MaximumDeleteBatchSize>250</MaximumDeleteBatchSize>
<MaximumCommitMilliseconds>250</MaximumCommitMilliseconds>
<ParallelizationDegree>30</ParallelizationDegree>
<MaximumRetryDelayMilliseconds>5000</MaximumRetryDelayMilliseconds>
<RetryCount>5</RetryCount>
<Encoding>utf-8</Encoding>
</Options>
</Solr.SolrWriterSettings>
If you are not utilizing SearchStax for any reason and find yourself in a situation where you need to create your own SOLR Web app (not in a virtual machine), and deploying SOLR via a virtual machine is not an option, you may encounter issues such as SOLR 503, 504, or 500 errors. Behind the scenes, SOLR is attempting to acquire a port on which it can initiate, but due to an inability to do so, these issues persist. After 3 to 4 attempts, SOLR should start successfully. However, if it fails to come up and run, you may attempt to resolve this by removing all the files located in the following directories. Open KUDU, CMD, and navigate to the locations specified below:
C:/Home/site/wwwroot/bin> rm *.port
C:/Home/site/wwwroot> rm log C:/Home/site/wwwroot/server/logs> rm log
C:/Home/site/wwwroot/server/tmp> rm -rf *dir
C:/Home/site/wwwroot/server/tmp> rm -rf *properties
By deleting these unnecessary files, you can free up space on your server, which has proven effective in multiple scenarios.
When making configuration changes for both SOLR and XConnect in an Azure environment, it's necessary to approach the process with careful consideration and attention to detail. By leveraging the resources provided and ensuring in-depth reviews of the environment-specific requirements, these changes can be efficiently and effectively executed to optimize the overall system performance.
By following this comprehensive approach to SOLR and XConnect configuration in Azure, the system can be fine-tuned to deliver optimal performance while ensuring a smooth and efficient operation.
Tip:
You might see this error while making changes.
Index dir 'C:\home\site\wwwroot\server\solr\sitecore_analytics_index\data\index/' of core 'sitecore_analytics_index' is already locked
Comments
Post a Comment