Job Description

The Max Planck Institute for Intelligent Systems (MPI-IS) conducts research to advance intelligent systems that perceive, learn, and interact. Through fundamental research across computational, physical, and social dimensions, we shape artificial intelligence and robotics to advance knowledge and benefit society.



We are looking for a



Student Assistant (HiWi) m/f/div LLM scaling laws



At Social Foundations of Computation



, we build scientific foundations for machine learning and artificial intelligence in the social world. To chart and implement a society’s norms and expectations, we start from concepts and work our way towards applications. Challenging existing problem formulations when necessary, we think through how the use of machine learning distributes societal resources and opportunity. Computational tools to critically evaluate - and possibly contest - algorithmic systems and their impacts are a key component of our work. Our ultimate goal is to promote a positive role of artificial intelligence in society.



About the project



We are building free-form question-answering datasets to study whether answer-matching and generative scoring yield smoother, more predictable scaling of LLM capabilities than multiple-choice metrics. The work spans data curation, validating correctness of benchmarks, and ambiguity audits—with lightweight tooling to run evaluations and visualize scaling behavior.

Relevant Papers:

1.



Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive?



2.Answer Matching Outperforms Multiple Choice for Language Model Evaluation



3.Pretraining Scaling Laws for Generative Evaluations of Language Models



Roles & Responsibilities

Your responsibilities may include:

Build on top of existing codebase. Convert existing MCQ datasets (across diverse domains) to freeform format using language models. Run large-scale evaluations across different model sizes (pretrained checkpoints), analyse results, and chart scaling curves. Optional: Make a small web UI for browsing responses of models and verifying answer correctness manually if needed. Contribute to a project that is planned for conference submission.

Education & Experience

Requirements:

Good coding skills in Python Experience with data processing (pandas, regex, jsonl), version control (Git), and LLM tooling (vLLM). Have solid experimental rigor: logging, ablations, seeds, and ensuring reproducibility Nice-to-have: Familiarity with pretraining, scaling laws, MCQ benchmarks, evaluation/metrics, annotation pipeline design.

Most importantly, we are looking for people who can iterate fast and use AI-copilot coding tools like Cursor/Claude Code/etc. to significantly speed up implementation time/learn new concepts on the fly and get results more quickly.

Our offer:

Students without a Bachelor’s degree will earn €13,80 per hour. Students holding a Bachelor’s degree will be paid €14,40 per hour. An initial contract will be offered for up to six months. Please do not hesitate to apply anytime, if you are interested in working with us. Contracts are given per semester.

Application:

To apply, please upload a motivational letter, a CV an references as one PDF file to our



application portal



. You are also welcome to send links to code or writing (GitHub, papers, blog). Any questions regarding the position should be forwarded to Nikihl Chandak at



nikhil.chandak@tuebingen.mpg.de



. If you are excited about building the datasets that make LLM progress more realistically measurable and predictable, we would love to hear from you!



The posting is open until filled.



The Max Planck Society is committed to increasing the number of individuals with disabilities in its workforce and therefore encourages applications from such qualified individuals. The Max Planck Society strives for gender equality and diversity. Furthermore, the Max Planck Society seeks to increase the number of women in its workforce in those areas where they are underrepresented and therefore explicitly encourages women to apply.