Elastic Workplace Search: search through your enterprise (cloud) applications
One of our clients asked us to research if it’s possible to include their SharePoint and Confluence content in the enterprise search application we build for them, which was very difficult. Since December, with the 7.5 release, Elastic has included Elastic Enterprise Search in the Elastic Stack. This triggered me to dive a bit deeper into this solution.
Note: This blog covers my view on version 7.6.0 and at this version Elastic Workplace Search is still in beta, which means it is not considered production ready yet.
Let me start by saying that the products in the Elastic Enterprise Search suite are part of the Elastic Platinum license. This means they’re not free to use. However, it’s possible to start free trials for each product which will last a month. In this blog I will walk through the Elastic Workplace Search product. First I’ll explain what it is about. Then I’ll cover the administrative side with integration of sources, user management, permissions & access control and settings for search/relevance. After that I will walk you through my view on the client side. I will show how the web client looks like and talk about other clients. Then I will finish with my honest opinion about the product.
What’s Workplace Search about?
By acquiring Swiftype in 2017, Elastic got ahold of their search solutions: Enterprise Search, Site Search and App Search. Recently Elastic has renamed Enterprise Search to Workplace Search and all three products are now part of the broader Elastic Enterprise Search suite. This blog is about the Workplace Search product. As said above, one of our clients asked us to investigate the possibility of including SharePoint and Confluence content in their search solution. According to Elastic, Workplace Search provides easy to use, powerful search across all the tools used throughout your organisation. It is an all-in-one solution to search through your company’s enterprise (cloud) tools in a single place. It allows you to quickly connect to multiple sources and automatically index all content into Elasticsearch to make it searchable. Searching can be done through the web application which comes with the suite, but other clients are also in the making. More on that later.
The Administrative Side
Source Integration
Workplace Search can integrate with all kinds of (mostly cloud based) sources. It has built-in connectors which can synchronise the content periodically. From what I’ve seen it does this every two hours and it is not (yet) possible to change this. All currently available content sources require you to create an OAuth App within that content source. For this OAuth App you need to create API credentials which you then need to provide to Workplace Search and after that the content will automatically be indexed in Elasticsearch. All of this is well documented in the Workplace Search Guide. Note: this means you need administrator privileges within the source you want to configure. The applications shown in the following image are currently supported: 
There are currently more connectors in the making for more sources. These are a few sources which we can expect to connect with in the future:
- Other Office365 apps (Yammer etc.)
- Rails Docs
- Box
- Evernote
Beside all connectors for these applications it’s also possible to connect your own source and index content from a custom application by configuring a custom API. Custom APIs are push only, so synchronisation of documents should be done from your own application. Each custom API source can have a custom schema. The below image shows how the management for a custom source looks like: 
When adding a new custom API, Workplace Search provides you with an Access Token and a Key. These credentials are needed to ingest data into Elasticsearch through Workplace Search. See the documentation on how to ingest data. Within the management section of your custom API you can create or alter field names and types for your schema. Ingesting data that matches the fields defined in your schema helps Workplace Search interpret and style the information. Ingesting documents which contains fields that don’t exist will create default fields of type text. It’s also possible to alter the display settings of your custom API data. With this you can change how the search results from your source will be presented. User management For the management of user there currently are three options.
- “Standard Mode”. With this mode all users are managed within Workplace Search. Users have to be invited from an administrator within Workplace Search (requires a SMTP server for emails).
- “Elasticsearch Native Realm”. With this option the users are managed within the Elasticsearch layer. Users and their role mappings can be created/altered through the Elasticsearch API’s or using Kibana.
- “Elasticsearch SAML”. New since version 7.6.0 is the option to configure Elasticsearch SAML, which delegates user management to third-party authentication providers and identity management platforms like Auth0 or Okta. You must use Workplace Search’s Role Mapping to coordinate user attributes with Elasticsearch.
The following image shows how the mapping of roles looks like in Workplace Search: 
As you might see, user attributes can be used to map users to Workplace Search roles and groups. Sources can be shared with groups to give users in these groups access to a specific set of sources. For example, someone from HR or marketing would not benefit from getting search results from GitHub. If the GitHub source is not shared with these groups, the search results from users within these groups will not include GitHub items. The following image shows the group management page: 
Permissions & Access Control One of the problems we ran into ourselves when researching Sharepoint/Confluence connectivity for our client is that those application have a complex permission/access structure. There is no way you can retrieve this and therefore cannot map this to your users within your search engine. You have to do all the mapping yourself which could be a lot of work. There could be sensitive content in those applications, which should not be visible to all users, but still you want the users working with that sensitive content be able to find that as well. Workplace Search doesn’t seem to have solution for this problem either, except to let every user connect their private accounts (private sources), but if you have a lot of users that will be very bulky. How this works in Workplace Search is that an administrator can set up sources for the whole organisation to use. These are called organisation sources. As said in the previous section sources can be shared with groups. If a user is not in a group where the source is with, the user will not be able to search in this source. Note that the content that is synchronised in organisation sources is everything that is accessible through the account with which it is connected. Connecting it with a master/administrator account might make private documents visible for the whole organisation within Workplace Search, because permission/privilege/role settings are not synchronised with the content. Document or object level permissions are not available (yet?) for sources with predefined connectors. Therefore make sure to connect the source with a special search user which only has access to documents publicly available for the organisation. Every user can connect their own sources that only they can use. These are called private sources. Only sources for which the connector has been configured in Workplace Search can be added as a private source, since you have to give access to the OAuth App created for the connector. Users might add their own private GitHub source or private Google Drive, for example. Its content will only be available for that user. This could be useful when you have a lot of private documents which you want to include in Workplace Search. For custom API’s it is possible to configure document level permissions. These are applied at index time. For each document you are able to define allow and deny permissions. Deny permissions take precedence here. A big side note here is this is only possible when users are managed within Workplace Search or through the Elasticsearch Native Realm. Managing permissions is done via the Workplace Search Permissions API. This can not be managed in the UI. Read the documentation about this topic if you wish to know more. Search Settings As for options to tune relevance, there currently are not many. Right now the only way to influence ranking is to prioritise sources within groups. You can make certain sources more important than others for a group. You can see how that looks like in the below image: 
Since the product is still in beta and more relevance tuning options, such as synonyms and boosts, are present in the other applications in the Enterprise Search Suite (App Search and Site Search) I expect those will be available later in Workplace Search as well.
The Client Side
(All images in this section are taken from Elastic) Web client For the client side of Workplace Search, they’ve built a web client which comes with the suite. They wrote a searcher’s guide which you should show the users before they start using Workplace Search. This is to make the users aware of all the possible features and how to use them. Let me first start with the good things about the client. It includes facets for filtering, time filters (past week etc.), option to sort on relevance or recently updated and has highlighting in the search results. When typing your search query it shows you your search history, which is synced in your account. You can click the “view in …” icon next to a search result when you want to go directly to the external source. When clicking on the search result itself it shows a preview in a sidebar with an excerpt and other metadata. In that way you sometimes don’t even have to leave Workplace Search to get the information you need. See the below image of how the search result page looks like: 
In my opinion the most notable feature is the natural language syntax. This syntax will let you search as if you were talking to a person, such as “issues assigned to Daniël in Jira with status in progress”. It recognises parameters in the query (they turn blue) and toggles and filters your results automatically. See the earlier mentioned searcher’s guide if you want to know more about this. A little impression on how this looks like in the search bar: 
Now about the bad things. First it lacks proper autocompletion. It only gives you search history or suggestions for sources after you type the word “in” or “on” (natural language syntax) and then the first letter of a source. It seems to have some form of toleration for typos, but I get no feedback about that. Maybe a feature like “did you mean” would be nice. The next thing is there is no option for UI customisation whatsoever. Other clients Next to that Elastic is working on more types of clients, such as native mobile apps, Slack integration and a browser plugin. This browser plugin will be useful while browsing through one of your connected cloud applications. It will provide you with relevant information from other connected sources on the fly. What this means is that, for example if you are browsing in Salesforce and you’re on an account page, you get information from other sources that could be relevant to this client. See the below image: 
In case you want to build your own client, this is not possible right now. They haven’t made the search API public yet. Elastic is planning to release this with GA, so you have to wait until then.
My Conclusion
In its current state I would not recommend Workplace Search. I believe it’s missing some essential features for a good search experience. No relevance tuning options, not any type of suggestions (except source autocomplete), no customisation to the client UI (except for the layout of custom source results) and no search API for a custom client either. There’s also the fact that a platinum license is needed. Unless you already have a platinum license for other Elastic solutions, I would not consider to purchase a platinum license just for this product. What I was really hoping for in this product is that they found some solution for the permission/access mapping. Unfortunately Workplace Search runs into the same problems we did. Still, Workplace Search has a lot of potential and I’m curious on how this product develops. Will definitely keep an eye on it.
Try it yourself
