Below are the steps I've taken to reduce my abandoned search rate from a high of 62% down to a low of 26%:
Step 1: Workflow
When we launched our portal in 2007 (Plumtree G6) our monthly abandoned search rate average (for the top 100 searches) seemed stuck in the 60% area. Horrified that our users were struggling to find content we identified that the titles and descriptions of documents going into our KD could be improved so we implemented the out of the box workflow for all documents added to the system as well as announcements posted to the home page.
This might have shaved about 5% off our abandoned search.
Step 2: Disable Full Text Search
Unsatisfied with a 55% abandoned search rate we began digging more into the features of the search tool to try to uncover what else might be causing such a high rate of failure. Our second step (which made sense at the time, although now I question) was to disable full-text search. The thought process here was that the search results were cluttered and that instead we should be focusing on appropriate titles and descriptions.
Since the out of the box portal interface is lacking the ability to dynamically utilize custom meta-data fields during the file upload process, we never adopted a policy of using other meta-data fields. Our search results are based upon a 96.77% weighting for "Name" (document title) and a 3.23% weighting for "Description".
I can't say that we saw a measurable change from removing the full-text search and now there are days when I think we would benefit from turning it back on, or at least begin using meta-data fields.
Step 3: Thesaurus
At this point we were still hovering around 50% abandoned which is still an F in my book so our next step was to make modifications to the thesaurus to account for acronyms, synonyms, and stemming - which the search tool was never able to handle natively.
Again this had a largely unnoticeable impact on search results:(
Step 4: Best Bets
At this point we were grabbing at straws and began to investigate the best bet feature which allows admins to cherry-pick the order of content in search results for identified terms. For those that have been on G6 you should recall a huge problem with best bets during this version in which a best bet which shared the first 2 letters as another best bet would be ignored. Mind boggling how something like this made it through QA and into a release, but that is another issue. Sadly the gaping flaw in the best bets prevented us from seeing much benefit here, and made it seem like we were chasing out tail trying to figure out why they weren't having the dramatic impact that we had expected.
Fortunately, in the 10.3 release the issue with best bets was resolved and you can now have nearly identical keywords that each work! This fix had a huge impact on our abandoned metrics and we were able to triple the number of best bets we had and knock our abandoned rate down from the mid-50's to the mid-40's overnight.
Step 5: Adaptive Layouts
The 6.5 upgrade was a pretty big step for the evolution of the portal in terms of search because it introduced the concept of adaptive layouts whereby a beginner coder could make a few simple modifications to an HTML file and dramatically alter the UI for the search results tool. Unfortunately the adaptive layouts were only half-baked in this release and we determined that upgrading wouldn't give us much benefit, so we held off until the 10.3 release which resolved our issues. Modifying the search results interface was a giant leap for us in terms of usability. It gave us the ability to take the ugly out of the box interface and turn it into something better which we designed based upon a combination of goggle, bing and Facebook. I made a blog post about the changes we did to the search results already, so I won't bother wasting your time again on the details. Suffice it to say, we got a lot of positive feedback on this. But from an abandoned search result standpoint it had little impact.
Step 6: Work with Support
I felt like progress was being made but felt like I was reaching the limits of what I could do with the tool out of the box. Search refinement sounded awesome but would probably require implementing an alternate search tool and didn't seem to be in the cards for our organization, especially after the announcement that WebCenter Interaction was being end-of-lifed by Oracle.
At some point I broke down and created a support ticket with Oracle to try to get to the bottom of our abandoned rate problem. Digging through our issue I was informed that Analytics ONLY looks at items stored in the Knowledge Directory (aka document repository) to determine the abandoned rate. So even though search indexes about two dozen object types, the only way to NOT count as an abandoned search is if you click on a "Documents in the Directory". Rather than throwing my hands up at this point I inquired about modifying the PEI or whatever tool it is that triggers the abandoned rate. Unfortunately I learned that it would be more involved than necessary and would have to find another solution.
Step 7: Revisit Best Bets
Armed with a better understanding of how the product works I went back into the trenches and yanked every non-document out of our best bets. Aware that there was a need for folks to access communities, videos and other objects that the KD doesn't natively support I created vanity URLs at the web server level and populating them into the KD and added them to the best bets also. This has been another of the biggest impacts on our abandoned rate and has probably accounted for a 10-15% drop for us.
Step 8: Look at Your Reports and Do Something
This past month our abandoned rate has dropped to 26.1% and I feel a great deal of weight has been lifted from my shoulders. I'm well aware that this number doesn't mean that folks are necessarily finding what they need, or that our system is good (I know that it has huge flaws) but at the same time you can't just sit idly by with an abandoned rate over 50%.
The overarching tool that helped identify our gaps was the top 100 search results report.
Each month I copy and paste the report into a larger spreadsheet that I keep and I run some simple formulas to identify problematic search terms. Armed with that I then revisit our best bets and thesaurus, as well as dig into the KD and examine titles and descriptions of files to see if they can be improved. When I find gaps or excess of search results for particular terms I reach out to our content managers and remind them to either add some content, or to remove outdated content.
Today we have 380 best bets and 4,670 documents in our Knowledge Directory.
My goal is to continue working with our content managers and improving our site with the tools provide to get to a <20% abandoned rate.
Step 9: Run Extra Queries
Analytics was a noble idea that languished in the BEA/Oracle days. Fortunately all the data you need is stored in the database and with a few extra queries you can get some meaningful data.
Query 1: Show me all documents that have been viewed less than 50x
SELECT CARDID, POPULARITY, HITCOUNT, LASTHIT, LASTMODIFIED_T
FROM plumdbuser.PTCARDSTATISTICS
WHERE (HITCOUNT < 50)
Query 2: Show me all documents that were created before 2010
SELECT NAME, CREATED
FROM plumdbuser.PTCARDS
WHERE (CREATED < '1/1/2010')
No comments:
Post a Comment