First Steps – Salesforce and Sitecore Integration

First Steps

By nature, I am extra cautious when I take my first steps on any project. This comes from experience, when a task is in front of you, many of us get tempted to jump and start working things out. I have a different take on this and even if I do burn couple of hours in the beginning, I would not change this style for no one or nothing! When you take your first steps right, when walls or falls happen, you are confident that you can get through them because you know that you are on right path.

So, on one of our projects, we decided that a tight integration with Salesforce would be a huge add on for this growing team. Now, the decision is clear that integration is needed, now, few key first decisions that should be made to ensure we will follow the right track to achieve our key goals.

Key Goal: We have a bunch of forms on the website that collect variety of information from end user based on purpose of the form. The stakeholders wanted to ship this collected information to Salesforce.

That’s a very abstract gist, I know. Not always would you have a dedicated Business Analyst that would document every single detail. Sometimes, you have to work with just a skeleton. Now, lets jot down the first set of questions:

  1. What Connector should we use to ensure we have the tight integration between Sitecore and Salesforce?
  2. What version of Connector should we be using?
  3. What does this Connector do on a very abstract level and would it help us reach the key goal noted above?

Now, lets answer each one of those first before we jump deep and go crazy. Which we will trust me. I did run across quite a few walls and was able to successfully break or crack enough to get by, I will share that side of journey with you all as well.

But, first things first, answer to above key questions. So, for the first one, my boss, Liz Spranzani has a great collection of blogs which highlights Connectors that are available and which ones to use in which scenarios. Salesforce offers a whole stack of products and so does Sitecore, so, it is important to get your thoughts straightened out, so, you make wise and correct decisions.

Here are the blogs for your reference. I made my choice to be to use Sitecore Connect for Salesforce CRM because that fits all my boxes based on subscriptions the client has in regards to Salesforce. Now, your answer could be different, I am hoping its the same, so, my journey will help you. 🙂

Now, for the second one, folks who know Sitecore know that this question is very important to be answered because of just the way Sitecore product life cycle has been. To confirm this, I checked the compatibility KB article, but, unfortunately, it is not always up-to-date. So, I actually loaded a support ticket to understand what version of connector should I be using given my Sitecore version, got confirmation that I should be using: Sitecore Connect for Salesforce CRM 2.1.0 . Bottom line is latest and not always the right approach especially when you are dealing with multiple systems.

So, on a very abstract level the connector gives you ability to push various objects from Sitecore to Salesforce and vice versa. Most important one of course that most of us will be interested in would be Contacts. But, along side of Contacts, it can also ship Tasks, Events and Campaigns.

You nailed your first steps, Now, what to do next? Installation! Sitecore does have good documentation on Connector Installation and setup, but, like always there is few gaps and some helpful tips to get you over few humps. I will cover those in my next blog.

Do not underestimate the power of making right choices, take your own time and do not let anyone make them for you if you are the one who is going to own the implementation.

Mysterious Experience Profile Error

I saw a lot of blogs and bumped in to online links about an issue below, but, turns out every case is unique based on setup. The issue I was facing, I could not find a solution out there, but, yeah I cracked it. What does that mean? Yeah, you guessed it right, blog it, so, some one else can save their half day and enjoy a nature walk or something. 😉

So, I was trying to hook up Sitecore and Salesforce together, I will get another blog rolling soon on my learning, frustrations, happy moments noting that experience. But, for this one here, just important to know one of the prerequisites for shipping contacts from Sitecore to Salesforce, first I need to ensure I can see the contacts on Experience profile. All was well on my local Sitecore installation which for reference is version 9.1.1

But…. Experience profile was broken to say the least on upper environments. On Experience Profile UI, you will see something like below. Booooo, not helpful at all!!!!!! Sitecore really need to display some meaningful errors here especially if it is non production license.

Once I got in to servers and starting checking logs, I could see below errors

URL https://authoring.mysite.com/sitecore/api/ao/v1/contacts/search?&pageSize=20&pageNumber=1&sort=visitCount desc&Match=*&FromDate=null&ToDate=null

Exception System.NullReferenceException: Object reference not set to an instance of an object.
at Sitecore.Cintel.Endpoint.Plumbing.NegotiateLanguageFilter.OnActionExecuted(HttpActionExecutedContext actionExecutedContext)
at System.Web.Http.Filters.ActionFilterAttribute.OnActionExecutedAsync(HttpActionExecutedContext actionExecutedContext, CancellationToken cancellationToken)
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Filters.ActionFilterAttribute.<CallOnActionExecutedAsync>d__6.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Filters.ActionFilterAttribute.<ExecuteActionFilterAsyncCore>d__5.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Filters.ActionFilterAttribute.<CallOnActionExecutedAsync>d__6.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Web.Http.Filters.ActionFilterAttribute.<CallOnActionExecutedAsync>d__6.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Filters.ActionFilterAttribute.<ExecuteActionFilterAsyncCore>d__5.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Filters.ActionFilterAttribute.<CallOnActionExecutedAsync>d__6.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Web.Http.Filters.ActionFilterAttribute.<CallOnActionExecutedAsync>d__6.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Filters.ActionFilterAttribute.<ExecuteActionFilterAsyncCore>d__5.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Controllers.ActionFilterResult.<ExecuteAsync>d__5.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Filters.AuthorizationFilterAttribute.<ExecuteAuthorizationFilterAsyncCore>d__3.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Filters.AuthorizationFilterAttribute.<ExecuteAuthorizationFilterAsyncCore>d__3.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Filters.AuthorizationFilterAttribute.<ExecuteAuthorizationFilterAsyncCore>d__3.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Dispatcher.HttpControllerDispatcher.<SendAsync>d__15.MoveNext()

Now, the above error had so many interpretations on web, but, led me no where , I then looked at XConnect logs to see if I can find deeper issue if any and I find the below error on XConnect logs

Sitecore.XConnect.Operations.FacetOperationException: Operation #1, ReferenceNotFound, Contact, Classification
2020-01-29 13:51:40.269 -05:00 [Error] Sitecore.XConnect.Operations.XdbSearchOperation`1[Sitecore.XConnect.Contact]: Sitecore.Xdb.Collection.Search.Solr.Failures.SolrResponseException: {
“responseHeader”:{
“status”:400,
“QTime”:12,
“params”:{
“fl”:”id”,
“cursorMark”:”*”,
“json”:”{\”query\”:\”(x_type_s:(\\\”ContactDataRecord\\\”) AND *:*)\”,\”sort\”:\”facets.engagementmeasures.mostrecentinteractionstartdatetime_dt desc,id asc\”}”,
“rows”:”20″,
“wt”:”json”},
“error”:{
“metadata”:[
“error-class”,”org.apache.solr.common.SolrException”,
“root-error-class”,”org.apache.solr.common.SolrException”],
“msg”:”sort param field can’t be found: id”,
“code”:400}

at Sitecore.Xdb.Collection.Search.Solr.SolrClient.EnsureSolrSuccessStatusCode(HttpResponseMessage response)
at Sitecore.Xdb.Collection.Search.Solr.SolrReader.<ExecuteQuery>d__14`1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Sitecore.Xdb.Collection.Search.Solr.SolrReader.<GetSearchResults>d__13`2.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Sitecore.Xdb.Collection.Search.Solr.SolrReader.<SearchContacts>d__10.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Sitecore.Xdb.Collection.Repository.<SearchContacts>d__11.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Sitecore.Xdb.Collection.RepositoryCountersDecorator.<SearchContacts>d__8.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Sitecore.XConnect.Service.RepositorySearchInvoker.<Execute>d__7.MoveNext()
2020-01-29 13:51:40.269 -05:00 [Error] [“XdbContextLoggingPlugin”] XdbContext Batch Execution Exception
Sitecore.Xdb.Collection.Search.Solr.Failures.SolrResponseException: {
“responseHeader”:{
“status”:400,
“QTime”:12,
“params”:{
“fl”:”id”,
“cursorMark”:”*”,
“json”:”{\”query\”:\”(x_type_s:(\\\”ContactDataRecord\\\”) AND *:*)\”,\”sort\”:\”facets.engagementmeasures.mostrecentinteractionstartdatetime_dt desc,id asc\”}”,
“rows”:”20″,
“wt”:”json”}},
“error”:{
“metadata”:[
“error-class”,”org.apache.solr.common.SolrException”,
“root-error-class”,”org.apache.solr.common.SolrException”],
“msg”:”sort param field can’t be found: id”,
“code”:400}}

at Sitecore.Xdb.Collection.Search.Solr.SolrClient.EnsureSolrSuccessStatusCode(HttpResponseMessage response)
at Sitecore.Xdb.Collection.Search.Solr.SolrReader.<ExecuteQuery>d__14`1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Sitecore.Xdb.Collection.Search.Solr.SolrReader.<GetSearchResults>d__13`2.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Sitecore.Xdb.Collection.Search.Solr.SolrReader.<SearchContacts>d__10.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Sitecore.Xdb.Collection.Repository.<SearchContacts>d__11.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Sitecore.Xdb.Collection.RepositoryCountersDecorator.<SearchContacts>d__8.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Sitecore.XConnect.Service.RepositorySearchInvoker.<Execute>d__7.MoveNext()

Below is the order of things I did to see if that will fix my issue, obviously no luck!!!

  • Ensured XConnect url is working properly that was provided on Connection strings
  • Rebuilt all indexes from Control Panel , ran successfully
  • Ensured SOLR url given on connection strings config works
  • Ensured NewtonSoft.Json.dll has correct version per Sitecore expectation
  • Restarted all Sitecore Windows Services + App pool recycle on CM and XConnect site
  • Also tried deploying all my marketing panel stuff – why not right? lol

Almost gave up, that is when I usually create a support ticket and wait for couple days before re-working on the issue. I did do that, but, in this case, I was having some more energy left, so, I kept debugging and kept digging- Paid real close attention to solr error above, I saw that the json was:
“json”:”{\”query\”:\”(x_type_s:(\\\”ContactDataRecord\\\”) AND *:*)\”,\”sort\”:\”facets.engagementmeasures.mostrecentinteractionstartdatetime_dt desc,id asc\”}”,
I fired up my solr console and entered this breakdown on query window and yep I could replicate the error. Now, I changed the sort field which was the one Solr did not like to _uniqueid and it worked. Why would it work with _uniqueid and not with id? Ringing bells? It did for me and for refresh of memory, I pulled open Sitecore documentation for my version on Solr setup instructions here
It says below – “repeat on all indexes”

But, XConnect code seems to fire Solr query with sort param as id, so, is xonnect UI incorrect? I was not sure and needed to confirm if Sitecore does not want us to do step #3 noted above on XDB indexes. So, I ran populate managed schema on my local and guess what all indexes have unique id modified to _uniqueid except for xdb related indexes. I mean this was still theory, I needed confirmation from Sitecore and my colleague bumped in to another set of documentation where there was NOTE, I could not believe my eyes when I saw that. So, yeah, my theory was correct and I had to login in to my not so favorite SSH Linux Solr server and hand edit the managed-schema file under both xdb and xdb_rebuild conf locations to have unique key as id and also replaced the settings line on schema to have id instead of _uniqueid to ensure all is swapped back. The NOTE that blew my mind and also comforted that I am on right path, screenshot below. May be Sitecore has to update documentation on the other spot to add this *****Important**** note to ensure folks do not change this on all indexes.

Now, once I did that and restarted Solr, Experience Profile is happy, there are no records just yet, but, no error either. yayyyy!!! On to my next challenge now. 🙂

New Search Experience New Challenges

If you think you have seen it all using a specific technology such as Coveo on your Sitecore implementation, think again. Every implementation is different, every client is unique and every challenge YES is a special beast. 🙂

With this brand new Coveo implementation we just delivered in two important phases had its own set of problems we had battle and solve. In this post I want to share a snippet of each and how we tackled each of it.

  1. Sitecore Architecture Limitation/s: When you are adding another layer of implementation on top of stood up system there is always challenges. It depends on how Sitecore was built. Was it built with Experience editor working correctly. Latest versions of Coveo highly depends on this assumption. We had similar issue on this, so, we created a content item added all presentation components we need and copied the presentation over to standard values of concern. We had another issue to battle, Requirement was to show some Sitecore items on Coveo Interfaces as results, but…They don’t have presentation and hence no Link or what so ever. We had to add a pipeline to make that happen and generate apt URL for these items that we want to send the user to.
    We injected a new processor that would generate the URL of specific items in coveoPostItemProcessingPipeline section. Index was now showing up with all the correct URL’s
  2. Variant Implementation : Like with most of commerce implementations, we had what were called base products and variants under each base products. We wanted to showcase variants as beautiful swatches that end user can explore if they are interested. For this client, it was very important to show special variant when a special property was selected from Facet available. To make this happen which data should be tied to index on which type of item was very important to analyse and acted upon based on experience that we were after. After that, it was pretty straightforward to implement variants leveraging grouping on Coveo Query Pipelines.
  3. Price Filter Issues : So, again, every customer wants their Price look and behave differently. Some like Slider, Some like Range and in our case it was just basically a string. This was done to adhere to their original implementation which they like. But, since it was multi language and multi site we ran in to challenges on how to have different fields on Index and how to use the different ones per language/site per context. This is because same field for all currencies would pull incorrect values as Facet Options.
    As a workaround, we loaded which field to Facet on based on Context. This resolved our issue of inconsistent values displayed on Facet. Another issue we faced is Custom Sort, since it is string it was hard to make Custom Sort work through Data Attributes on Coveo Rendering properties. We had do a patch on Javascript side on event preprocessResults to make this requirement meet. It has an object called groupByResults which can be modified to achieve specific order we need.

There were some more interesting challenges we encountered while deploying and testing post Go Live, I will cover those in my next blog. 🙂

Symposium Summed Up

Just in time for Thanksgiving, I finally got my thoughts all wired about Sitecore symposium this year. It is super hard to sum up the whole experience in one word, but, if I must I have to say it was ‘Marvelous’.

I must say Sitecore and their team really works hard to come up with a theme that connects and has an impact on all of us. While I walked away from Symposium, It was just not the learning, social and happiness for me this time. It was something beyond, it was the Connection I could feel literally. I was very much inspired by the ‘Human Connections in a Digital World’ theme this year. There are always some sessions that align with the theme that are very engaging. This year was no different, some key notes that were the key for goose bump moments on my end were the below

  1. Talk by Platon on Powerful Portraits was so mesmerizing, for a second I forgot that I am in Sitecore symposium. It was not one bit related to Sitecore, but, was much needed to look beyond, answer some questions you would not dare to and understand these personalities we all know from different perspective.
  2. Talk by Dr. Rana el Kaliouby on Emotional Analytics. This was by far my favorite session in terms of keynotes and main stage ones. It is just such an eye opener, it shows that we need to continue to think how to better our amazing systems and technologies we use in our daily lives to add a bit of something that will make the systems think for us and like us. It is simple in terms of logic and almost felt like Yes, we know it and we agree it. But, we are not there yet, kind of surprising. Take away from this is – Emotional AI is not a future thing, we need to act now and remember what makes us US. Would definitely hope Mr. Elon Musk will integrate this Emotion capture in Tesla. 🙂
  3. How can I not mention my two favorite people that I long to see on stage. Jake Johanssen and Sitecore CMO Paige O’Neill. When ever Jake dawns the stage you can hear every one just being happy, he makes us all laugh so much, love him for that and also guess what I could take a picture with him as I bumped in to him. Check us out here ! All smiles. 🙂 Paige is a true inspiration for me, I had an opportunity to meet and greet with her last year at my Symposium talk about Empowering Women. She is an awesome person with amazing stage presence.

Now, coming to break out sessions, I would be unfair to myself if I don’t start with mine lol. So, I will start there. Wow!!! I cant put in words how grateful I actually am to be up on that sparkling stage in-front of so many wise, enthusiastic people. Really feel humbled and proud at the same time to be able to deliver what we planned. You can check more details about my session here. Truly rewarding experience overall!

Below are some of the other sessions I could make it to between practice drills of my own session and other important keynotes. I must say I enjoyed all of them and learned a thing or two. You can now download the session materials for all of the below including mine from connect

  1. Not Impossible: Accessibility for all : This session is a continuation to the keynote by Mick Ebeling , CEO of Not Impossible Labs. His session last year on main stage moved me to tears. This time I was well prepared. 🙂 The co-presenter of this session are Liz Spranzani , SVP, Development , Verndale and Ross Lucivero, Managing Director, LA, Verndale. My two favorite people from my own company. It was very informative session and almost made us all question why do we even produce sites that are not Accessible. If we consider every one of us as equal beings, we should strive to show equality and accommodation on the sites we build as well.
  2. Marketers’ guide to getting started with Sitecore Cortex personalization suggestions : This session was like the most crowded session I attended this time around. People were literally standing and still paying attention. Kudos to Una Verhoeven, Managing Partner, AmpleEdge to get the audience so thrilled and excited about Cortex personalization. Key takeaways from my end were Cortex can be leveraged to provide personalization suggestions and it is important to avoid personal bias. Like anything else, honing is important to ensure personalization acts as expected.
  3. Sitecore AI (MVP Session): I can not give away a whole deal of information since this was MVP only session, but, it was very exciting session that almost got me dreaming in a good way. My next thing for sure.
  4. ‘O Solr mio! : Jeremy Davis totally nailed this one. I was literally looking forward to this one, kind of bummer it was on last day because with Sitecore going SOLR as default option starting 9.0+ it is immensely important we know our options and we know setup for production can be done in a simple and correct way , so we are rest assured on Website Search. 🙂 Love it!!! Very information and as always scripts were released right after the show.

All in all, this symposium was important affair for my personal growth, I walked away stronger, happier and fully satisfied. Best of all, I got to take Sitecore Pillow and Hedgie back home. My daughter owns both right now, but, yeah, glad I could get it back home. I have never been to Chicago, so, really hoping to see how Sitecore rocks it next year.

Netflix Next With Sitecore Cortex

Cover for Session

This year’s Sitecore symposium was a nostalgic experience filled with a lot of emotions on my end. I am really honored to have been up on the famous Sitecore stage and for the opportunity to be able to share our story with my peers and beyond. I can’t thank Sitecore team enough who believed in our idea and thought it was worth a share. This story would have not been this beautiful minus my partner in crime @RichardLeivaA, so, official thank you to you for being who you are and sticking through the tricky bits and buts of our path.

For those who could not make it to our session. We want you to hear our story as well. Below are the slides for your reference. Leave a comment or Slack me if you need any additional information and resources.

Github Url of Demo: https://github.com/richardl1/netflixnextsymp2019

When Performance is a Lock

Lock

Performance is usually on back of mind when I do typical chores of a Lead/Architect on any project. Unfortunately, with tighter timelines and supreme focus on getting the tasks done with out even us realizing if often takes a back seat. But, we all need to strive for excellence and the only way you would be able to deliver a quality solution is with a legit key if performance were a lock. 🙂

Performance has that good spot on a quality solution checklist as none of the users like waiting in this fast pace world. I was taking a look back in my blogging and found this one here where I wrote my heart out on some tips that can improve your Sitecore solution in general, this time around I will be adding some personal learning on this current Coveo for Sitecore I am part of where performance is a lock 😉 and in my quest to find a key, I actually found a bunch of things you could do to top off what I already knew. Again, thanks to awesome @Coveo team who always supports the goals and challenges of the integration project and trust me they are never the same, with a different client comes different set of challenges. But, that is the fun right?

Now, enough of the story, lets jump right in to the details. I do not like to repeat what Coveo already recommends, so, I will point you all to right resources where I can.

  1. Case 1: When what you have on Sitecore is simply not enough!
    You have couple of options here to battle this. Which path you would choose usually depends on type of data that you are after. What is the latency tolerance on the piece of data? Is context involved? Do we fetch from third party or is it drawn from business logic based on existing data on your sitecore? When you answer these you could pick the option right that would actually affect your performance in a good, bad or ugly way depending on what you choose.
    If the additional information is user context driven then the processor Coveo offers is way to go. You can find more information here
    If you can live with latency or is tolerable and logic is quite complicated to place in processor noted above, then, computed fields is way to go. You can read more information here, this usually comes with an additional step to ensure your data is fresh especially if your computed fields are drawn from a third party source as Sitecore publish might not refresh some of your data. The additional step is to actually rebuild a specific area of your content tree in a scheduled manner. With Coveo 5, you can use the Index API and fire a special crawler. See more info here, do not forget to configure your new crawler first to keep the scope to what you care about as we all know indexing is a quite expensive operation. If you did not know how to, here is the link to do the same.
    So, choose your path wisely and that will put rest of performance in check.
  2. Case 2: So, yeah, you cant show everything
    This is like almost always right? There is bunch of content on Index, but, we cant show everything in every case. May be some items are only for special group of users or may be we want to show only content items of specific template. I did this like so many times in the past, but, never looked at it from performance lens, when Coveo mentioned we could improve performance with some simple configuration settings on the filters we plan to use, it made be re-read this document here to find what I just skimmed through in my last read. There are so many good/easy settings you can use to improve performance on these filters. Read more here
  3. Case 3: Oh boy, you have loads of data on Index
    I guess any enterprise Sitecore solution would have ton of content, but, we do not want all of it on Coveo Indexes, when you think about it, what you would show on Coveo search interface to user would need to be a page, so, why bother pushing items to index that do not have presentation, right? In most cases that is true unless you have a very different content strategy and link modifications on Sitecore, in these cases we cant exclude all items minus presentation as a default. You can read how to do this here
    In our case, we could not do this, so, had to clean up some not needed data on Index via inclusion/exclusion rules of templates. You can read more about this here

    As we roll along with implementation, I am sure we will learn more and more ways that Coveo provides to keep the performance in check. Again, it was so greatly satisfying to keep learning new things with every implementation. Hope this helps some one who is looking for more ways to enhance performance of their Coveo for Sitecore implementation. This is just a scratch on surface per me, what I would recommend is to work with Coveo team for project specific tricks you could do to ensure Coveo works and behaves as cool as it should be.

See the magic – Coveo Recommendations

If you have not gotten a chance to review my other posts that I wrote up based on our experience implementing Coveo recommendations from the scratch, I would strongly advise going through them first.

Post 1 – Was all about visual clues

Post 2 – The real deal

In this post, now that we have gotten all the set up done, I will focus on how to actually get some recommendations to show up to ensure we apply styles, test the filters out and basically yeah see a working version of recommendations. It was not actually that straight forward, but, thanks to couple hands and also huge thank you to @vbernard to help give some awesome tips that helped us force recommendations when everything else failed.

First, when we have the rendering up on the presentation on all our product detail templates where we wanted to show recommendations from Coveo, we started by visiting lot of pages that made sense to the user flow. I had couple of hands, no kidding here, so, basically I read few posts to understand how Coveo recommendations work internally, though it is a black box, it is fairly easy to understand the basic logic behind how it works. So, we attacked it in such a way that it is bent to give us some results.

This is what we did and it worked on our lower environment and started sending recommendations back, you can also choose to use your QA to fire off some tests and automated requests to do the same. We did a bit of both actually – manual go to set of links that emulated various levels of our product pages and an automated hit to several urls as well.

In a nut shell, Coveo recommendations looks for patterns based on history of the end user and suggests what might interest the user next based on the history and matches that it can find from the recommendations model. It is also important to ensure recommendations model has enough data by adjusting the training settings on the ML model. We tried couple different settings there and rebuilding the model after we visited bunch of pages, circling around the products we care about. Finally! We see the below on our recommendations component

We got lucky to be able to test this on lower environments due to multiple hands and effort by the team to emulate what we could, so, we can test it thoroughly with filters on hand. Oh! In regards to filters, we actually needed lot of context based filters, so, we approached this in two fold process –

  1. Adding a processor via config to coveoProcessParsedRestResponse, this essentially would add in some additional meta data per result that we need for queries and filters. There is some real good example and documentation loaded here . We followed similar path, but, added our own business logic to load context data needed on our filters
  2. Next up, we added a JS query filter as documented here and used a bunch of expression builders based on all the data we loaded either on processor or via the recommendations view such as current item ID or some miscellaneous information we needed per business requirements on what to show on recommendations and what not to.

Now, I am a true believer of back up plans, if our hardwork to emulate and get the recommendations engine to send us a few recommendations had failed, then, we would have applied styles and do other tests via a hack provided to us by @vbernard from Coveo. Here are the steps to make that work:

Step 1: Add a new pipeline

Step 2: Push some featured results to this pipeline

Ignore the cool stones and take a peek at the featured results

Step 3 : Now go back to ML models and add a recommendations model

Thats it! You should now see the recommendations forced and you would probably see the featured results on your recommendations. Do note though that you should start really small/basic on your view if you are trying to do this forced result approach. In our case, it was kind of hard to use this approach and test what we needed to as we really lean on context for most part. But, probably it might work on basic use cases.

The Real Deal – Coveo Recommendations

In my last post, I focused on the visual clues. If you have not already, please read the first post on coveo recommendations here

In this post, we will focus on actual steps once you did decide that Coveo Recommendations will serve your requirement on hand. Coveo, as always does awesome job with the documentation on how to use recommendations by following the steps noted below. Since Coveo does such a good job at documentation, I am not going to repeat myself here and refer you all to check out this page here

Now to highlight our experience when we followed the steps noted on the Coveo documentation, we learned the below on each step along the way:

Step #1 – Currently, there is no way to actually see the page view events from Coveo Administrative tool, but, you have two check points to quickly note if page views are being captured. Quick screenshots below for reference

Step #2 & Step #3 : It is one of the most straight forward steps and after you quickly add the pipeline and model, you should be able to see them up on Coveo Administrative tool

Go to Models under Machine Learning section

Query Pipelines
You should be able to click on Pipeline, go to machine learning tab and click on edit for the model to see the settings on training set, that is one of the most important settings to ensure you set to appropriate values based on your data set.

Step #4 & Step #5: We figured that this is actually not needed in our case. It was working fine with out doing any additional set up, turns out this was just for sanity or may be older versions of Coveo for Sitecore, not sure. But, was not needed for us.

Step #6 : So, this step once we have our own datasource item, we added Coveo Recommendations to our presentation on all templates that needed to show this component. When we added this component, we could see a new call fire off on XHR requests that would go fetch recommendations. It also started maintaining history of the user views and sent that along with request. We encountered an error here because one scenario where special characters persisted in Title field of the page would cause server error as noted here . We submitted a PR for this scenario and the fix was included in latest release of JS framework

That is it and the last step was publish and we figured we should start seeing recommendations right away. But, it is not that simple. 🙂 In my next post, I will go over the various tricks we did to get the recommendations to show on our pages where we inserted this component.

Additional note here, we did want to show only product recommendations, so, to avoid additional filter additions in our case, we chose to add content type on our page views which we can then utilize on our filter on model to show only specific content type like below.

As you can see the specific filter allows you to pass in a string for content type which is actually set on Sitecore on the rendering properties for Coveo Page Analytics
This should be picked on datasource set on Coveo Page View Analytics rendering

Next up, I will share how we did the filters, forced recommendations, a quick tip shared by @vbernard when everything else fails on how to trick the system to actually send in some recommendations and any other issues we encountered while implementing recommendations using Coveo.

Visual Clue for Coveo Recommendations

Coveo offers a bunch of cool ML functionality. As some of you may already know, growing and providing ML relevance in various streams of website journey is one the main goals of Coveo and where it is heading as an Enterprise product. Coveo has some very good documentation on what their ML components are and how to get started with these here

But, given all that, it is still hard to put a finger on to whether a specific ask or requirement could fit in and benefit with the use of Coveo ML component. So, I decided to blog on our journey beginning from a thought to reality. It was a lot of fun and some additional twists due to lot of context specific filters that were needed to ensure we only show what is truly relevant to the context to top the recommendations that Coveo thinks are apt based on user journey.

So, in this blog I would like to focus on when to know and understand that the current requirement on hand could be a good candidate to implement Coveo solution instead to get the powerhouse technical bits that Coveo is real proud of. Now, it all started when for the very first time I heard a high level requirement around showing recommended products on a e-commerce platform based on user history in combination with couple of Sitecore based fields. My first immediate thoughts and concerns when I hear lot of content management effort would be :

  • Data Integrity – When lot of manual steps are involved, it is usually hard to have a tap on data integrity even with proper workflows in place
  • Tedious and frustrates Content Authors – The more there are to steps, the more it keeps getting on the pile of TODO for the team of CA. Probably the team might never get to it if the priorities change

Now, lets focus for a second on first requirement, so, it is based on user browsing history. If we are confident on the patterns and can learn from how user navigates between products, do we then have a better way of projecting what the user may be interested in rather than making our CA team work hard? Yes, it is an absolute Yes!. Coveo Recommendations component does exactly that by tracking and learning constantly as user explores the site, the model is then capable of returning recommendations.

So, say, there is a scenario where user interactions should not matter and probably the relation ship data between products is coming from a ERP or CRM based system? In this case, since we do not need relevance power of Coveo and is just mostly business logic driving a finite result set, this would be a perfect opposite use case where you could pull it off with few simple SOLR queries to get what you need.

It is very important to get the decision finalized before you dive deep and actually use a cool component Coveo has handy. So, if you are going forward with implementing Coveo Recommendations component, this is a good place to start and understand the steps needed to build one. In my next blog as part of this series, I will explain each one in detail, later will move on to context based filters, hive components to use, sample results to test some HTML with.

You want to use a CDN with Sitecore huh?

First things first! Do we need a CDN at all when Sitecore is your platform of choice. Not really, you can actually do pretty awesome good in terms of performance with good caching, scheduled publishing, uploaded good imagery and rock solid code with few best practices followed. But, what if you have to use a CDN? There will be various situations in which you may have to due to business reasons or may be the audience that is very spread out across the globe or if the site is down right very media heavy.

Various questions pop up when folks bring up CDN. Where exactly is this going to sit in the bigger picture? What is the plan to use for CDN for? What do we do on Sitecore to make this happen? Are the assets on CDN? What happens behind the scenes? Now, it is more than likely that most of us already know answers for this. So, I will not bore you all explaining what are questions that need to addressed and understood when thinking about CDN with Sitecore.

Instead, what I would like to do is provide a quick comparison cheat sheet based on various features and capabilities when you have made your decision to use CDN with Sitecore. Recently, I was researching and finding information about what is the best way to integrate CDN with Sitecore in most efficient and easiest way possible. Couple years ago, I remember that integrating with CDN was still considered considerable amount of work. But, with Sitecore 9.1+ it should be real simple if of course the basic OOTB functionality that around CDN is enough for your business needs. I also explored one more option which is not bad either if you have time to make the legacy code work with newer versions of Sitecore.

Option 1: CDN Connector

  • Reference URLhttps://github.com/NTTDATA/SitecoreCDN
  • Multi Site Support – Yes
  • Sitecore 9.1 Support – Unknown, but, some of our team suggested they did make this work on 8.2 and hence conceptually with that fix that we have to get this working on 8.x should probably and hypothetically work for 9.1 as well. I would recommend a POC if you are going to use the connector. As you can see from source url, the code base is very outdated almost 6 years ago. But, turns out with few fixes it may still work and does the job well. 🙂
  • Media Versioning – Yes. it does stamp the media with an incremental value based on date of publish which would help knock cache and have good versioning in play
  • File Versioning – yes. Based on documentation, it does support file versioning for static assets as well. But, this is something I highly recommend testing quickly before confirming that in fact it does this fine.
  • Other advanced features – It actually does more than this set and if you are curious of full functionality you should read this great resource https://github.com/NTTDATA/SitecoreCDN/blob/master/NTT%20DATA%20Sitecore%20CDN%20Connector.pdf
  • One important thing to note which was pointed out by my peer is that the connector actually checks to see when to load a resource from CDN vs when not to. For instance, if there are goals set up on a download of specific pdf for instance, it should be loaded from Sitecore media library and not from CDN. The connector handles this, but, obviously the code might be subtly outdated. Easy to fix this though if not working and I totally think this should be done if personalization is actively used in your sitecore instance. You can check that additional logic here if curious.
    https://github.com/NTTDATA/SitecoreCDN/blob/master/Code/Providers/CDNMediaProvider.cs

Option 2 – Sitecore Media Library CDN

  • Sitecore 9.1 actually comes with a special package to enable CDN
  • Reference urlhttps://doc.sitecore.com/developers/91/sitecore-experience-management/en/sitecore-media-library-cdn-overview.html
  • Multi Site Support – No. If you pay attention to config file in the package that Sitecore provides you will see couple of settings that enable CDN use on media library. But, these are not site specific and are global. Hence, this does not support different settings for different sites.
  • Sitecore 9.1 support : obviously, Yes. 🙂
  • Media Versioning – Yes
  • File Versioning – No. You can see an added note of no such capability here, scroll to the bottom of the page. https://doc.sitecore.com/developers/91/sitecore-experience-management/en/sitecore-media-library-cdn-overview.html
  • Screenshot of settings for reference in CDN.config which is the only thing on the package provided on downloads section

Now, there is no wrong or right way. It depends on your requirements and what would be most efficient path to choose. It is great that Sitecore actually has some option now to plug and play and use CDN OOTB in some level. Only hope is that OOTB we get more settings and flexibility to choose CDN configuration that comes OOTB than to implement some thing to fit needs that has become a new normal these days. I am sure we will get there soon. But, for now, pick and choose. If these do not fit your needs, feel free to make something custom. Custom solution actually takes me back to really old versions of Sitecore that needed custom solution to fit in CDN in-between media library, server and end user.

Coveo new GUID generation issue

Thursday last week, one of my team member ran in to an issue which was preventing XHR request to fire off to Coveo and get the results back. He struggled to overcome the issue almost whole day long and at that time as his lead it became my responsibility to help solve the issue. I was actually quite doubtful about it and unsure if I could help solve or not, but, as a lead, rule is you can not pass on fear to your team, you have to no matter you feel pass confidence that – ‘Yes, we will get over this’. Challenge Accepted!

Here is a sneak peek of the issue. So, in older versions of Coveo and in more fond memory on Legacy framework implementation, each interface element had a unique ID that you would need to pass from presentation details via properties on your rendering. If you do not do much about it, a unique identifier will be generated when you create a new content item that has the presentation of Coveo embedded. With hive, it is actually not so different, pretty similar, only the properties are on data sources. The issue on hand is that, on the interface view HTML, the @Model.ID was actually coming back as ‘$GenerateNewUniqueID’ instead of DOM Element ID noted on presentation. You can see this below on the screenshot.

Error on JS for generating unique ID.

So, as with any issue, it becomes very important to replicate the issue on local especially when you can not get your brain juices flowing on call on tough problems like this. See the steps I did to crack this thing

  1. Why would Coveo not replace this in first place? It should automatically happen. My team mate stumbled upon few Q&A and checked the pipeline related that does this behind the scenes to ensure that is in place and hooked properly.
  2. We checked the presentation again very carefully and datasources to ensure DOM ID is present and is unique
  3. Ensure this is copied over properly to Final Layout on Sitecore as well and indeed there is a correct data source being passed in presentation.
  4. Then we ran an ugly experiment to actually replace standard value as it seemed like Coveo for some reason was taking that. No luck, still same issue.
  5. Re-installed Coveo For Sitecore package relating to our Sitecore version and published everything. Still Nothing!
  6. This was working just fine on our UAT server, so, just to try our luck ported over all packages from UAT related to bare bone Coveo templates, layouts, etc., Nope! Did not work.
  7. Ran out of all options!

One little detail from my team mate sparked an experience from the past. He said when he installed a package that has just one item while he was experimenting did something and though the XHR request was still having lot of issues, he could now see the ID replaced now. That reminded me that in the past when we ran in to weird issues where presentation was not updated even if the presentation did look right on Coveo end, an IIS Reset had solved all problems. Went ahead and tried that and knocked out browser cache just in case and boom, everything started working.

I should have probably done IIS reset first before trying anything, but, with more knowledge you gather you would want to explore that rather than blind belief that a restart or reset would solve things. 🙂

This whole debugging experience got me thinking if I was not a Software Engineer, I would have made one awesome FBI Agent for some reason. It almost felt like I was connecting from past and paying attention of little detail on hand to solve the issue. Quite a few memories of great crime investigation series flashed in my brain. lol

Gather Content & Sitecore

I have to say I truly love any tool such as Gather Content ( https://gathercontent.com/) that has all the tools for ensuring right sort of marketing content is prepared for a website. It has a good and fair game of workflows, content templates and statuses for each to assure that you need everything that can give more power to you on a pre-CMS lifecycle. With new sitecore integrations, Style labs should be the way to go for some of this chores ahead of Sitecore’s role.

Now, when you and your team does decide to use a product such as Gather Content, it is very important to start strong and correct issues right in the begining of content load if you ever wish to use the connector ( https://gathercontent.com/resources/how-to-use-gathercontent-for-sitecore-projects) provided by Gather Content to pre-load some content on to Sitecore when the templates are ready up there. You do not have to match the architecture of your Gather Content templates to Sitecore because the connector does provide some cool mappings that you could spin up. What you should rather be focusing on from day one –

  1. What is it that I would like to store on Gather Content
  2. What specific content areas/items do I want to migrate on to Sitecore.
  3. How can I help ensure that the data loaded on GC is website ready – Meaning all image having good dimensions, any design directions or important design guidelines to follow.
  4. Keep it simple and organized — That is the reason you have a system like Gather Content – Focus on painful and heavy content areas.

Also, keep in mind the shortcomings of a system like gather content. Though they do their job close to perfect on providing the tools needed for the goals of such system. Automated content migration need not be a seamless, dreamy world because as always we are talking about two unique systems which expose different data types, hierarchies and tools. So, nail down what you care about on this migration. Establish an question equation that when Sitecore is up and running, that should be the tool for content entry. Gather Content is a pre-CMS tool and should be used and respected for that.

Few Caveats as of this writing. I am pretty sure Gather Content Sitecore connector will mature to accommodate some of this very frequently needed tasks on hand while working with Sitecore.

  • Is the connector Sitecore 9.0+ compatible 
    • Yes, per their documentation and video tutorial. It is very much compatible.
  • How do items get created when the location where we chose to put items is a bucket? Does that affect your connector at all or it should work as expected 
    • GC team will get back to me on this shortly and I will update the blog or post as a comment as soon as I receive feedback
  • Where do we maintain source listing if any on Gather Content 
    • This is a manual task for now, in the future there will be options for more dynamic tagging – GC suggests adding a character code to the title of an item to “tag” it
  • Do you have a demo available that showcases porting of multi list complex items from Gather Content on to Sitecore. 
    • general link – Curious on how this is setup on Gather Content and ported over to Sitecore
      • Need to check with GC team – Could potentially put this off and only load on Sitecore to minimize not needed effort
      • GC: No, wouldn’t be able to work with multifaceted integration, more basic for template mapping – GC will follow up with integration partner
    • For multilist, Should datasources noted on Sitecore template first mapped and entered properly so Gather Content can then have that mapped correctly.  
      • I  guess is there a way to ensure the data sources are replicated on Gather Content as a feed for multi list to ensure they are in SYNC?
      • Again very name specific and no way at this time to source from somewhere else other than defining on template all options again
  • Images â€“ We want a very organized tree on Sitecore for media.  Is there is a set up configuration we can do for to ensure this is streamlined. 
    • Need to check with Gather Content team
    • It seems like it does respect the source set on Image field when importing the images on to sitecore from Gather Content based on Template mapping, but, confirmation might help
    • GC: Confirmed, you can choose a location for porting images – GC will still double check with the integration partner
  • Shareable Content 
    • Say, we have shareable content across the templates of Gather Content.  Is there some kind of picker that ties between different entities on Gather Content? 
    • OR 
    • Should we expect to have all the mappings for this shareable content across multiple sections of different templates.  What happens to duplication at the point of using Connector in this case? You should go this route, but, do understand there is no association that can be provided between two sets of Gather Content Items. It is all manual at this time.
  • Any special filter we could do and ensure that when Connector does port a list of items that match specific query go to different location on the Sitecore Tree. 
    • Not present at this moment as standard functionality, but, can be extended. If I do receive details on how to do so I will add a comment to this post as and when I get more info.

RTF Configurations OOTB Sitecore

rtf

Okay there comes a time when your business analyst is going to ask you what does Sitecore provide by default in terms of rich  text options.  And there are actually couple of blogs out there that are informative about this.   Check my favorite one here  https://markstiles.net/blog/2011/3/15/rich-text-editor-profiles/

But, it is only screenshots up there and though we are all intimate with those word icons, it is hard to actually know what they do unless we open up the editor and hover on each icon to ensure our assumption is correct. 

So, I built the below table as a handy reference.  Hope this helps!  Below are the list of RTF configurations,  a quick screenshot for visualization and list of functionalities it exposes to your authors.

Based on the features available per configuration, business analyst and front end team would be able to interpret how many custom configurations we need vs which ones from OOTB that we can re-use for our design requirements on modules in question. 

Also, a rule of thumb is to limit custom configurations between 3 to 5 based on number of modules and complexity of Sitecore implementation.  More than that would be a overkill per me. 

Default RTF Configuration 

Screenshot depicting default RTF Configuration

Options Available

  • Bold
  • Italic
  • Sitecore Link
  • Sitecore Media
  • Remove Link
  • Hyperlink Manager
  • Insert Symbol
  • Paste from Word – Strip Font
  • Undo
  • Redo
  • Find and Replace

Medium RTF Configuration

Screenshot depicting the options for Medium RTF

Options Available

  • Print
  • Find and Replace
  • Cut
  • Copy 
  • Paste
  • Paste from Word
  • Paste from Word  – Strip Font
  • Paste Plain Text
  • Paste as HTML
  • Format Stripper
  • Undo
  • Redo 
  • Insert Sitecore Link
  • Insert Sitecore Media
  • Remove Link
  • Insert Table
  • Insert Symbol
  • Insert Code Snippet
  • Bold
  • Italic
  • Underline
  • Align left, right, center, justify
  • Remove Alignment
  • Numbered List
  • Bullet List
  • Indent
  • Outdent
  • Strike Through
  • Sub Script
  • Super Script
  • Horizontal Rule
  • Show/Hide Border
  • Module Manager
  • Ajax Spell Checker
  • Help
  • Apply CSS Classes dropdown

RTF IDE Configuration

screenshot depicting options for IDE RTF

Options Available

Everything that is available on medium RTF configuration listed above plus a couple more advanced features

  • Insert Form element
  • New paragraph
  • Insert Date
  • Insert Time
  • Foreground Color
  • Background Color
  • Set Absolute Postion
  • Select All
  • Font Weight
  • Font Style
  • Custom Links
  • Zoom

Full RTF Configuration

Screenshot depicting Full RTF Configuration

Options Available

Everything that are available with medium RTF configuration noted above along with couple more advanced options

  • Insert Form Element
  • New Paragraph
  • Insert Date
  • Insert Time
  • Insert Code Snippet
  • Media Manager
  • Flash Manager
  • Foreground Color
  • Background Color
  • Select ALL
  • Set Absolute Position
  • XHTML Validator
  • Font Weight
  • Font Style
  • Custom Links
  • Zoom

Architectural Excavation on Integrations

coal mining

Hello to every one from amazing London!  Sad to say that though London is awesome, I do miss LA as I consider that my home.   The main aspect I miss not having to bundle up even in December. 😉

I wanted to share some new things I am learning at this moment.  Every project and role that you play, you tend to look at things from  a different lens and perspective.  Leadership and Architecture go hand in hand for the most part, but, Architect is known to look at things from a very high level to make decisions that are very important for a successful implementation.  

One such aspect of project development that needs such kind of initial research, series of gathering sessions to excavate both business and technology requirements is so much fun as it unveils things that were never thought of while we are in life cycle of  website development.    To be honest,  I get nervous when I can not visualize full picture and I strive to calm myself down by erasing some of the clouds that block my vision by being organized and asking lot of questions in  general.   When I see the picture (though not perfect or clean) I am then more confident and relaxed.  I wanted to share with you all my first hand experience as I learn a  new game being in sweet, yet, challenging place. 🙂 

  1.  Document a list of every single third party integration that you would need.  This should include big, medium, small.  Usually, the big ones are actual integrations with API of some sort.  Medium ones could still involve some template and module building on Sitecore, but, usually some third party script.  SOLR or Search you would use would also come under either medium or big based on level of implementation.  Small ones are typically just JS script injections in to either head section or body section based on specific requirements of the third party. 
  2. I like to call this step Identification path.  On each of the big ones noted above think about data you would like to be shown on your website.  When you merely start doing this step you might encounter questions that you never thought before.  I LOVE THIS step.  I urge every one to definitely do this. 
  3. Now, we are only talking about data because in my case data was utmost importance than any acquired functionality for instance.  Now, if your use case is more on functionality, you could pursue in that direction.  In this step, based on identified areas we determine if we need to consume the data we need by directly calling the API or by adding these data elements as actual Sitecore items.  The decision typically revolves around latency, performance of API and Search-ability.   Noting all this aspects and vision along side of good collaboration with needed parties you can land on decisions that make sense for your project. 

Having these decisions made before implementation is very critical in understanding the complexity, clear breakdown of base tasks that are needed to pull your implementation together and most importantly for me is the ability to be in my calm state is priceless. lol

If any of you need more information on any of the steps I listed or decisions you need to make based on outcomes, reach out to me and I can help you.  It was fun exercise for me and hoping you all will have some fun around this as well when you play the role of Technical Archaeologist.  Could this be a new role you think?  

Sitecore Workflows – Fun Little Surprises

workflows

Sitecore life is funny,  just when you think you know everything, there is something that shows up and suggests that there is always something new you can learn with Sitecore even in tiny section or feature when compared to Sitecore on the whole.   This has been proven time and time again. 🙂  But, I love it that way and I think surprises is directly proportional to fun.

I would like to share some of those Aha moments I have had with while working with Workflows.  Now,  in huge transformation projects such as say we are building a brand new site in Sitecore or not much of a legal compliant client for example we usually get away by building a simple workflow, more or less OOTB one that comes with Sitecore installation with few more commands based on requirements.  When you do that, there is not much you bump in to actually as Sitecore does the magic kind of fine in those instances.  Now, what if you get to work on workflows that are heavy complex, customized and awful lot of scenarios?  Guess?? You will learn and wait for it….and say Aha a whole lot more 😉

Here are few of those that I encountered –

Do you know TDS/Sitecore actually shares relationships/associations of roles in counter intuitive way? Do not get what I mean, let me expound a little.  Say, you have a role called Content Author and now you have this another role called Content Author Base.   You went ahead and added Content Author base as a member of Content Author.  Now, if you look at your definition file in TDS, you will notice something funny, you will see that Content Author item is not modified, it is Content Author base that is modified to add that association.

The second one is a bit of sad surprise actually I am just a tad bit sad that I could not achieve what I wanted easily and obviously workaround was much easier so I went right ahead.  So, if you have a custom comment template defined at workflow command level, the expectation is it would show up on workbox when the command is clicked both on item level and on bulk level.  So, turns out the version we were using did not have that configuration and all bulk operations did not show any comment.  Our comments are mission critical in our case, meaning we can not skip them as there is some decision making involved in each of these commands and states.   In later versions it seems there is a configuration that controls to show or not show comments on bulk operations ( I did not test this tough, kind of trusting sitecore support)

<setting name=”Workbox.SingleCommentForBulkOperation” value=”true” />

In our case though upgrade was not an option for this tiny use case, so, we decided to hide all the bulk buttons by replicating Sitecore.Shell.Applications.Workbox.WorkboxForm in to our own custom class and using this class instead on our workbox.xml

I tried hard to see if there is any way I can ask Sitecore to show my custom comment template rather than not showing anything at all or showing some custom parameters built on the same class I stated above.  There is a good stack overflow thread that talks about how you can show a comment instead, but, in my case it would not work as I do need a custom comment template to show up and I do not want to manually add it as suggested on the post.  If you are curious here is the post –

https://stackoverflow.com/questions/22947912/sitecore-workbox-does-not-prompt-for-comment-when-approving-multiple-items

I could have gotten away with this if a simple comment would work.   Another suggestion that Sitecore support offered was to pass in Default Comment Template on the workflow definition item level, as it turns out it is a good hack if you have one comment template that you need to show across the workflow.  In our case, it is specific to command and nope will not work.   I wish there was an easier than this, but, hey, we convinced the folks to ease some pain by choosing to just disable bulk commands, turns out they want the specific roles to not have these bulk commands and would like them to pay close attention to what they are approving, so, works in our favor.

If you are curious, how we hid the buttons,  again, takes us back to same magic class.  Pay attention to HideButtonsForCommand.  All this function would do is check against app settings of some sort and do not add buttons if command matches to our needs.

foreach (WorkflowCommand command in workflow.GetCommands(state.StateID))
{
if (stateItems.CommandIds.Contains<string>(command.CommandID) && !HideButtonsForCommand(command.CommandID))
{
XmlControl webControl1 = Resource.GetWebControl("WorkboxCommand") as XmlControl;
Assert.IsNotNull((object)webControl1, "workboxCommand is null");
webControl1["Header"] = (object)(command.DisplayName + " " + Translate.Text("(selected)"));
webControl1["Icon"] = (object)command.Icon;
webControl1["Command"] = (object)("workflow:sendselected(command=" + command.CommandID + ",ws=" + state.StateID + ",wf=" + workflow.WorkflowID + ")");
border2.Controls.Add((System.Web.UI.Control)webControl1);
XmlControl webControl2 = Resource.GetWebControl("WorkboxCommand") as XmlControl;
Assert.IsNotNull((object)webControl2, "workboxCommand is null");
webControl2["Header"] = (object)(command.DisplayName + " " + Translate.Text("(all)"));
webControl2["Icon"] = (object)command.Icon;
webControl2["Command"] = (object)("workflow:sendall(command=" + command.CommandID + ",ws=" + state.StateID + ",wf=" + workflow.WorkflowID + ")");
border2.Controls.Add((System.Web.UI.Control)webControl2);
}
}

I bumped in to crazy language specific security issues.  On few languages, I started seeing a prompt on content editor warnings section that I do not have access to this language and it suggested me to swap the language, so, I can continue editing.   This was insane as I knew there was no language based security that was intended nor there.   I went to my user/role and checked on access viewer to make sure I see full green for two important security flags when language is concerned.

What could be wrong, I first thought since my item’s owner was Sitecore\Anonymous in that language, I thought Sitecore was using sitecore\Anonymous security in which it made sense, but, nope, I changed the owner to some sane one and still same issue.   Finally, I cross verified what is on /system/languages and what was on my content item languages.  It seems some language specific cultures did not have the node in /system/languages and when Sitecore is asked to respect the configurations on read and write language flags and it can not find such item on the location for languages, it interprets as a “NO” to be on safe side.  That was exactly the problem on my hand, those missing language nodes in combination with Sitecore configuration was not working.

Last one, here it goes.  Have you ever wondered what can be done to take down Lock and Edit completely for few roles if needed.  I am sure there are multiple ways to approach this, but, when I dug deep in to how Sitecore is pulling which commands to show on Review tab under workflow panel, I bumped in to below piece of code which we had to modify to take down Lock and Edit.  Along the side, we also took down Content Editor Warnings to ensure we do not show a prompt of Lock and Edit as well.   Now, I am sure there could be other ways, but, the below is just one way.  I am usually against swapping pipelines, but, it is important and have to, I caution around least invasion and propose thorough testing.

So, for not displaying content editor warnings, you can inherit your class from the below and do what you got to do in that class.

<processor type="Sitecore.Pipelines.GetContentEditorWarnings.IsLocked, Sitecore.Kernel" />

For the workflow panel override to not show Lock and Edit under Review ribbon, make your own custom WorkflowPanel class.   Swap the core item to use this new Workflow Panel class instead of default one.   The core item is located under –

 /sitecore/content/Applications/Content Editor/Ribbons/Chunks/Workflow/WorkflowPanel