In a white paper a few years back, I opined that adoption of Virtual Digital Assistants platforms was being slowed down by the fact that building and maintaining its ‘knowledge’ was still a predominantly manual task and therefore expensive and slow. Over the last 20 years, many companies have adopted Virtual Assistants on their website, and some with a reasonable success, that showed consistent ROIs and improved customer satisfaction. However, Virtual Assistants never became pervasive during last decade. Many early adopters were excited about the possibilities, envisioned new uses for them, not just for customer support but also as hostess, marketing/product advisors. But one of the issues that muted this excitement was the inefficiency of the knowledge management process. These solutions relied on rule-based systems that required manual intervention to retrain the virtual assistants. While these systems did use machine learning for certain features, e.g., to adjust keyword weights or remember frequently used responses to queries, but the creation and modification of the dialogs was pretty much a manual effort. If in doubt, they accepted the alternative of just placing a static FAQ or search engine that indexed the knowledge automatically, even if the user experience was not nearly as good. This attitude has changed over the last few years, and companies are now recognizing the importance of the conversational user experience to retain customers. For the virtual assistant platforms still around, this could be a renewed opportunity for a breakthrough. The rule-based Virtual Assistants are better for businesses than the modern ML-based models, for two critical reasons: 1) amount and quality of data available to build reliable ML-based systems, 2) simplicity and predictability of rule-based systems. 1) Because to create a reliable ML model, a company needs lots of knowledge, data, or past customer interactions. Usually, the data available to businesses, unless you’re Amazon or Google, is limited to build reliable models. New models and algorithms requiring much fewer data to start training themselves are being experimented, but these are not commercial solutions yet.
2) The other reason that rule-based systems are still better than model-based Virtual Assistant is that a rule-based system is most predictable, it's easier to understand why the system is wrong or correct. To fix a problem, it’s straightforward as you can pinpoint the faulty rule in a matter of hours, if not minutes. In the case of ML model-based assistants, if the model is found to be giving wrong answers, it takes data scientists to fix the model, and that may take days because these are hard problems. So, your process is more automated, but you end up spending more time fine-tuning it. Also, in the absence of data scientists on your staff, this is not a viable solution anyway. So, if we appreciate the simplicity of rule-based Virtual Assistants, and if can we automate or semi-automate some the manual processes, specifically, the generation of the dialogs, we will have a best-of-both-worlds solution. Some of the recent advances in NLP and machine learning could help in this specific task while leaving the data flow and architecture of the rule-based systems intact. Using a hybrid model composed of a rule-based component augmented by machine learning monitored off-line by knowledge specialists. Machine learning algorithms have exploded, open source platforms like Apache Spark MLib, TensorFlow, Stanford NLP and Open NLP are being used everywhere. It’s in these open source systems, and new releases, with their classification, summarization, and language generation algorithms, that lays the answer to creating a more automated system that can build dialogs on-the-fly. This new system would provide an initial framework/dialog tree and allow the training of the Virtual Assistant organically through reinforcement learning. It’s a combination of knowledge classification, summarization, and language generation algorithms that the reinforcement learning builds the missing dialog prompts on the fly. Finally, the knowledge specialists promote new successful prompts into the dialog tree for future interactions. There have been efforts and new technology developed by new upstarts like Yewno and Viv Labs, which have built machine learning solutions that can generate hierarchical taxonomies or create ‘the code’ behind the dialogs entirely autonomously. This tells me that the technology or understanding of the problem appears mature today for hybrid Virtual Assistant systems and the semi-autonomous learning to become a reality. Comments are closed.
|
Categories
All
Archives
January 2019
|