Voice Command Recognition Bot to Automate Employee Meeting and Vacation Management

We suggested developing a voice recognition solution to enable fast and hands-free operation, and that suggestion was accepted by the client’s team with great enthusiasm. To ensure the ease of installation and usage as well as to prevent employees from cluttering their app stack, it was decided to develop a telegram bot, based on language model, that would recognize not just fixed commands, but spoken, conversational language in English and German (as the client team is bilingual).

The voice-powered chatbot allows employees to solve the following tasks:

• schedule a meeting with the specification of the meeting day, time, duration, title, and participants
• reschedule the meeting or adjust its title and participants
• cancel the meeting
• get information on the meeting: available slots for a certain day, planned meetings for a certain day, and their details (time, participants, title)
• book vacation of any type (paid, unpaid, sick leave, special day-off, etc).
• reschedule or cancel vacation
• get information on the vacation: how many days are available, how many days are booked and when.

To enable automatic spoken language recognition instead of rigidly fixed voice commands, our data scientist developed a range of data clusters, where each cluster was associated with a certain possible query from the above mentioned use cases.

Another task was to ensure the accuracy of voice command recognition. To attain this objective, our team implemented various spoken language and word recognition strategies and thoroughly trained the language model in different sound conditions – where volume, intonation, accent, and other characteristics varied.

Let’s get an overview of a user flow:

1. Аfter the activation of a telegram bot, a user can say a meeting- or vacation-related phrase in English or German.
2. The bot identifies the language automatically, then identifies primary and contextual keywords in the user’s phrase. Based on these keywords, the bot determines the complete user query. The query may be either action-based (e.g. cancel a meeting, book a day off), or information-based (e.g. provide available amount of paid vacation days next month).
3. The bot passes the user query to the HRIS system and meeting scheduling software. If the query is action-based, the target tasks get completed, while if the query is information-based, the target information gets back from the source systems.
4. The bot informs the user one’s the query is completed (if action-based), or provides the requested data (for information-based queries).

Thanks to the integration with the client’s HRIS and meeting scheduling systems, all changes are applied and all data is updated in real time, preventing irrelevant results and collisions of meetings and vacations of different employees.