PLAICraft Multimodal Gameplay Dataset
Time-aligned video, audio, action data collected from multi-player Minecraft World, and pre-encoded latents for every gameplay session, generated from actual files in an S3 bucket. This is an initial release of a small subset out of the 10,000 hrs of data that are security check passed, more will be released in the future.
PLAICraft
Minecraft
multimodal
game AI
Emboddied AI
video
audio
action
sqlite
SQLite session DB containing the mouse and keyboard action data from the Forge mod, with session metadata
application/octet-stream
audio_in
Player microphone input (WAV)
audio_out
Game-audio output (WAV)
event_log
Event log (JSON) containing the system level key loggings
positions
Position stream (JSON) (not being used)
videos
Screen-capture video (MP4)
enc_audio
Encoded audio latents (HDF5)
enc_video
Encoded video latents (PT)
All sqlite files in the dataset
path
filename
player
session
All audio_in files in the dataset
All audio_out files in the dataset
All event_log files in the dataset
All positions files in the dataset
All videos files in the dataset
All enc_audio files in the dataset
All enc_video files in the dataset