Local LLM-Powered NPCs with Llama.cpp, Mistral7B and StyleTTS2

Dec 24, 2023 ·

In this article, we discuss the use of Local Language Models to create Non-Player Characters in Unreal Engine. This technique is used to allow for more realistic behavior when interacting with these NPCs. Local Language Models are trained on data from a specific game or environment to generate conversation and behaviors that can be used to simulate life-like conversations and actions within a virtual world. Through the use of this technology, developers are able to provide a more immersive experience for their players.

The primary method for using Local Language Models in Unreal Engine is through the use of the Unreal AI Framework known as Behavior Trees. This framework allows for the creation of Artificial Intelligence peers which can interact on a deeper level than traditional scripted NPCs. Through the use of Behavior Trees, developers are able to give their NPCs a range of behaviors and reactions that they might encounter while playing the game. This is accomplished by providing the AI systems with dialogue and action files containing information about the character’s emotional state and responses.

Additionally, developers are able to make use of Natural Language Processing to further customize the dialogue and interaction that an NPC may have with a player. By utilizing NLP techniques, developers can create context-based conversations which are more natural and believable. This can provide a much more engaging experience for players, as they will be interacting with NPCs that are capable of responding appropriately to questions and comments.

Finally, Local Language Models can be used to create different types of interactions between NPCs and players. For example, they can be utilized to simulate conversations between two NPCs, or between an NPC and the player. Additionally, these models can be used to simulate conversations between multiple NPCs, such as a group of friends, or a family. This can create more interesting stories and scenarios for players to explore.

In conclusion, Local Language Models can provide developers with the ability to create more realistic and immersive experiences in Unreal Engine. Through the use of Behavior Trees and Natural Language Processing, developers can create NPCs that act in a more life-like manner, making for an engaging and believable virtual world.