Historical development diary. Code examples are reproduced in their original language and reflect the project at the time of writing.
Published on 6 January 2024
The vision
As someone passionate about both history and education, I have always been bothered by how static and one-dimensional history education can be. Reading Martin Luther’s theological ideas is one thing, but what if you could actually have a conversation with him?
The idea behind this project was to bridge historical texts and modern interactive learning. By combining LLM technology with character animation, we created an AI avatar that does more than recite facts: it engages in meaningful dialogue about theology, reform and faith while aiming to remain historically accurate.
We also wanted to show that a project like this could be developed in just one day. We love a challenge.
Technical setup
- Game engine: Godot 4.3
- AI integration: OpenAI API with the gpt-4o-mini model
- Animation: Custom 2D sprite system with dynamic mouth shapes and expression transitions
- Primary sources: Curated collection of Luther’s writings and contemporary documents
- Configuration: Key management with environment-variable and file fallbacks
- Text display: Optimised character-by-character animation system with natural timing
Development journey
Phase 1: foundations and AI integration
One of our first challenges was finding the right “voice” for Luther. Initial attempts produced responses that were either too modern or too stiff and academic. The breakthrough came when we restructured the prompt to emphasise Luther’s role as a teacher and reformer.
We quickly built a simple system using the OpenAI API and added an abstraction layer to make switching between LLM providers and models easy. This allows the project to become more capable and cost-effective as models improve.
Some prompt engineering brought us to a reasonable level of authenticity. After experimenting, we found that prompting a teacher to play Martin Luther worked best. The system is less easily distracted by modern topics and guides the conversation back to Luther’s theological views.
Here is an example of the system prompt we use:
You are playing Martin Luther, the Protestant reformer of the sixteenth century. Stay in character while recognising that this is a historical portrayal. Your responses should reflect Luther’s theological views, personality and historical context.
Core character traits:
– Strong convictions about faith, scripture and salvation through grace
– A direct and passionate communication style
– Scholarly, yet able to speak to ordinary people
– Known for serious theological discourse as well as witty remarks
Animation commands are integrated into the prompt so that the LLM can naturally control the character’s expressions and movements:
Animationsbefehle:
[POSE=X] für Grundhaltungsänderungen
[EMOTION=X] für Gesichtsausdrücke
[GESTURE=X] für spezifische Bewegungen
[MOUTH=X] für Mundformen während des Sprechens
Verfügbare Befehle:
POSE: neutral, nachdenklich, lehrend, leidenschaftlich, entspannt
EMOTION: neutral, freudig, besorgt, ernst, leidenschaftlich
GESTURE: nicken, kopf_schütteln, nach_oben_zeigen, hand_winken, buch_referenz
MOUTH: A, F, I, L, M, O, S, U, idleA key technical decision was to implement a layered architecture for LLM integration:
# Basis-LLM-Client für gemeinsame Funktionalität
class_name LLMClient extends Node
# Spezifische Implementierung für OpenAI
class_name OpenAIClient extends LLMClientThe OpenAI client implementation includes configuration management:
class_name OpenAIClient extends LLMClient
const API_CONFIG_FILE = "res://config/openai_config.cfg"
const API_KEY_ENV_VAR = "OPENAI_API_KEY"
var _cached_key: String = ""
var _config: ConfigFile
func _init() -> void:
_config = ConfigFile.new()
_load_config()
func _load_config() -> void:
# Zuerst Umgebungsvariable prüfen
var env_key = OS.get_environment(API_KEY_ENV_VAR)
if env_key and env_key.length() > 0:
_cached_key = env_key
print("API-Schlüssel aus Umgebungsvariable wird verwendet")
return
# Auf Konfigurationsdatei zurückgreifen
var err = _config.load(API_CONFIG_FILE)
if err == OK:
var config_key = _config.get_value("api", "key", "")
if config_key and config_key.length() > 0:
_cached_key = config_key
print("API-Schlüssel aus Konfigurationsdatei wird verwendet")
return
print("Kein API-Schlüssel in Umgebung oder Konfiguration gefunden")We also created a template configuration file:
# openai_config.template.cfg
[api]
key="ihr-api-schlüssel-hier"
model="gpt-4-mini"
temperature=0.7
max_tokens=150
[system]
debug_logging=false
cache_responses=trueThis abstraction proved valuable because it allowed us to:
- Switch easily between different LLM providers
- Test different models, beginning with GPT-4-mini
- Provide a consistent interface for the rest of the application
- Manage API keys using several fallback options
Phase 2: animation system
The animation system developed over several iterations:
- First version: simple mapping of characters to mouth shapes
- Enhanced version: phoneme-based mapping with timing controls
- Current version: context-aware system with dynamic transitions
The animation manager now handles all aspects of character animation:
class_name AnimationManager extends Node
# Sprite-Referenzen
@onready var mouth_sprite: Sprite2D = $MouthSprite
@onready var expression_sprite: Sprite2D = $ExpressionSprite
@onready var pose_sprite: Sprite2D = $PoseSprite
# Animations-Status
var current_mouth_shape := "idle"
var current_expression := "neutral"
var current_pose := "neutral"
var is_transitioning := false
# Mundform-Texturen
var mouth_shapes = {
"A": load("res://assets/A.jpg"),
"O": load("res://assets/O.jpg"),
"I": load("res://assets/I.jpg"),
"S": load("res://assets/S.jpg"),
"U": load("res://assets/U.jpg"),
"M": load("res://assets/M.jpg"),
"F": load("res://assets/F.jpg"),
"L": load("res://assets/L.jpg"),
"idle": load("res://assets/Relaxed.jpg")
}
# Phonem-Zuordnung für verbesserte Genauigkeit
const PHONEME_MAP = {
"ah": "A",
"aa": "A",
"ae": "A",
"oh": "O",
"ow": "O",
"oo": "U",
"ee": "I",
"ih": "I",
"eh": "I",
"ss": "S",
"sh": "S",
"ch": "S",
"mm": "M",
"bb": "M",
"pp": "M",
"ff": "F",
"vh": "F",
"th": "F",
"ll": "L",
"tt": "L",
"dd": "L"
}
# Übergangs-Timing
const TRANSITION_TIME = 0.1
const MIN_SHAPE_TIME = 0.05
func _ready() -> void:
# Mit Standardzuständen initialisieren
set_mouth_shape("idle")
set_expression("neutral")
set_pose("neutral")
func set_mouth_shape(shape: String) -> void:
if not mouth_shapes.has(shape):
print("Unbekannte Mundform: ", shape)
shape = "idle"
if shape == current_mouth_shape:
return
if is_transitioning:
await get_tree().create_timer(MIN_SHAPE_TIME).timeout
is_transitioning = true
current_mouth_shape = shape
# Tween für sanften Übergang erstellen
var tween = create_tween()
tween.tween_property(mouth_sprite, "texture",
mouth_shapes[shape], TRANSITION_TIME)
await tween.finished
is_transitioning = falseThe text display system was optimised for smooth animation and natural timing:
class_name TextRevealManager extends Node
# Timing-Konstanten
const CHAR_REVEAL_TIME = 0.025 # Sekunden pro Zeichen
const WORD_PAUSE_TIME = 0.075 # Pause an Wortgrenzen
const PUNCTUATION_PAUSE_TIME = 0.15 # Pause bei Satzzeichen
const PARAGRAPH_PAUSE_TIME = 0.3 # Pause bei Absätzen
# Statusverfolgung
var _is_revealing := false
var _current_text := ""
var _revealed_text := ""
var _current_pos := 0
# Signale
signal text_revealed(text: String)
signal char_revealed(char: String, pos: int)
signal reveal_completed
func reveal_text(text: String) -> void:
if _is_revealing:
await cancel_reveal()
_is_revealing = true
_current_text = text
_revealed_text = ""
_current_pos = 0
while _current_pos < _current_text.length():
if not _is_revealing:
break
var char = _current_text[_current_pos]
_revealed_text += char
# Pausendauer basierend auf Zeichenkontext bestimmen
var delay = _get_delay_for_char(char)
# Mundform basierend auf aktuellem und umgebenden Zeichen aktualisieren
var context = _get_char_context(_current_pos)
var shape = animation_manager.get_mouth_shape(char, context)
animation_manager.set_mouth_shape(shape)
# Signale für UI-Updates senden
char_revealed.emit(char, _current_pos)
text_revealed.emit(_revealed_text)
await get_tree().create_timer(delay).timeout
_current_pos += 1
# Zurück zur Ruhe-Mundform
animation_manager.set_mouth_shape("idle")
_is_revealing = false
reveal_completed.emit()
func _get_delay_for_char(char: String) -> float:
match char:
' ':
return WORD_PAUSE_TIME
'.', '!', '?', ':':
return PUNCTUATION_PAUSE_TIME
'n':
return PARAGRAPH_PAUSE_TIME
_:
return CHAR_REVEAL_TIME
func _get_char_context(pos: int, window: int = 2) -> String:
var start = max(0, pos - window)
var end = min(_current_text.length(), pos + window + 1)
return _current_text.substr(start, end - start)Phase 3: educational features
We implemented a dynamic learning-objective tracking system that:
- Monitors conversation topics in real time
- Links discussions to predefined learning objectives
- Provides contextual prompts to guide the conversation
The learning objectives are structured around core theological concepts:
const objectives = {
"rechtfertigung": {
"title": "Rechtfertigung durch den Glauben",
"description": "Verständnis von Luthers Kernlehre der Erlösung allein durch den Glauben",
"keywords": ["Glaube allein", "sola fide", "Rechtfertigung", "Erlösung", "Gnade"],
"sources": ["unfreier_wille", "von_der_freiheit", "freiheit_eines_christenmenschen"],
"prompts": [
"Was meinten Sie mit 'Glaube allein'?",
"Wie erlangt man die Erlösung?",
"Warum waren Sie nicht einverstanden mit der katholischen Sicht der Werke?"
]
},
"schrift": {
"title": "Autorität der Schrift",
"description": "Erforschung von Luthers Betonung der biblischen Autorität",
"keywords": ["Bibel", "Schrift", "sola scriptura", "Wort Gottes"],
"sources": ["deutsche_bibel", "roemerbrief_vorlesung", "bibelkommentare"],
"prompts": [
"Warum haben Sie die Bibel ins Deutsche übersetzt?",
"Welche Rolle sollte die Schrift in der Kirche spielen?",
"Wie sollten Christen die Bibel auslegen?"
]
}
}Technical challenges
1. API key security
- Implementing several fallback options for storing API keys
- Environment variable (OPENAI_API_KEY)
- Configuration file (openai_config.cfg)
- Adding key caching to reduce file-system access
- Creating a template system for safe version control
- Handling missing keys correctly
- Adding debug logging for the key-loading process
2. Response streaming
- Developing a custom streaming solution for real-time text display
- Implementing character-by-character animation
- Optimising performance for smooth animation
- Implementing appropriate pauses at punctuation
- Allowing text display to be cancelled for a better user experience
- Creating a signal system for UI synchronisation
3. Resource management
- Creating a relevance-scoring algorithm for primary sources
- Implementing URL validation and security checks
- Developing a caching system for frequently accessed resources
- Adding type safety for string arrays and dictionaries
- Improving keyword matching with variations of terms
4. Animation system
- Fixing mouth-shape loading problems by switching from preload to load
- Fixing type mismatches in the animation manager
- Optimising sprite transitions for smoother animation
- Adding debug logging to track animation states
- Improving the timing system for more natural speech rhythm
- Implementing context-aware mouth-shape selection
Future development
1. Speech synthesis integration
- Exploring ElevenLabs integration for an appropriate voice
- Planning lip-sync improvements for spoken output
2. Extended animation system
- Implementing gesture blending for smoother transitions
- Adding an emotion state machine for more natural expressions
- Improving mouth-shape transitions through interpolation
- Adding more varied and dynamic expressions
- Implementing advanced timing algorithms for speech patterns
- Supporting stressed syllables and emphasis patterns
3. Educational features
- Developing a curriculum builder for teachers
- Supporting custom primary sources
- Creating a progress-tracking dashboard
- Implementing a more sophisticated relevance-scoring system
- Adding support for multiple languages and translations
4. Security and configuration
- Implementing encrypted storage for sensitive data
- Adding user-specific configuration profiles
- Improving error handling and recovery
- Expanding logging and debugging tools
Lessons learned
1. AI integration
- Careful API key management is essential
- Prompt engineering requires extensive testing
- Configuration management must balance security and ease of use
2. Animation systems
- Basic real-time animation is deceptively simple to implement
- Performance was not a bottleneck thanks to the lightweight system
- Type safety in Godot 4 requires particular attention
- Timing is essential for natural-looking animation
- Context-aware animation decisions produce more realistic results
3. Educational design
- Although we created a foundation for learning features, we did not have time to test them thoroughly with real users
- The dynamic resource system proved more valuable than expected, helping users explore topics more deeply in a natural way
- Balancing historical accuracy and accessibility requires careful consideration
- Primary sources need clear context and explanation to be useful
Conclusion
This project shows how game development technology and AI can be combined to create engaging educational experiences. The Martin Luther avatar demonstrates that history education need not be static: it can be interactive and personalised while pursuing historical accuracy.
Combining LLM technology, real-time animation and educational features creates a distinctive learning environment that engages users while seeking to preserve historical authenticity. There is still room for improvement, particularly in refining animation and testing the educational features, but the foundations for future development are in place.
Jonas Heinke