System with a visual LLM-based chatbot builder that automates customer support. The platform uses technologies and tools to process large data streams in real time.
The customer is engaged in distribution of telecommunication equipment, video surveillance and security systems. Detailed information about the client cannot be disclosed under the terms of the NDA.
The customer wanted to implement a 24/7 support system for their customers based on LLM. The customer set the task to develop an intuitive platform with a graphical interface for training and retraining of LLMs in case of new documentation release or in case of changes in the current documentation. It was assumed that existing services would interact with the trained models.
We developed a user-friendly visual interface builder specifically for creating chatbots. This interface allowed clients to create and customize chatbots according to their unique requirements without requiring extensive programming knowledge. In addition, our team worked on creating a bot rating system. This system allowed customers to leave feedback on the performance of the artificial intelligence-based chatbots.
We used FastAPI and aiohttp to create a high-performance REST. PostgresSQL was used as a database to store client data and some bot settings. Tensorflow was used to train and retrain LLM models, interaction with the built pipeline was performed via FastAPI API of the application. Ray was used to scale the application, as it is specially designed for such scenarios. Kafka and Redis were used to store and process data from users in real time. Prometeus was implemented to monitor the system performance.
Scrum was used to manage the development of the project using Agile methodology, which allowed to get a working prototype in the shortest possible time and gradually increase the functionality to the required for the customer.
The customer received an innovative tool for automating 24/7 support based on LLM, also the customer got an opportunity to retrain models independently due to the lined pipeline and developed UI for interaction with it. The customer’s clients, in turn, were able to receive 24/7 accurate instructions from the chatbot to interact with the customer’s products.
