简介

Flame: Advancing vision-language models in front-end development via data synthesis

English | 中文

• Introduction
• Features
• Installation
• Usage
• Dataset
• Model
• Contributing
• License
• Acknowledgements

Introduction

State-of-the-art models like GPT-4o, while powerful in generating code for webpage creation, fall short in meeting the dynamic requirements of modern front-end (FE) workflows. The code they generate is often static, lacking essential features like modularity, reusability, and dynamic behavior, which are critical for building scalable, interactive user interfaces. This leads to inefficient and incompatible code that deviates from best development practices.

To overcome these limitations, we introduce a comprehensive framework that includes a data synthesis pipeline, model training process, and evaluation suite, forming a fully integrated vision-language model (VLM) solution for front-end code generation. Using this framework, we developed Flame, a front-end language assistant with multimodal expertise, by integrating the Siglip Vision encoder and the Deepseek coder model for React code generation.

One of the primary challenges in developing a robust large-scale VLM for front-end development is the lack of high-quality image-text data. To address this, we propose an automated data synthesis pipeline that extracts, renders, and annotates self-contained front-end code snippets. This pipeline ensures the generation of large, diverse, and high-fidelity datasets, capable of producing both single-image and multi-image inputs, along with detailed image descriptions for improving visual chain-of-thought (CoT) reasoning. In this process, we leverage DeepSeek's API to integrate both the DeepSeek V2 and V3 models for dataset construction.

To evaluate Flame’s performance, we have established a comprehensive evaluation suite that measures three key factors: syntactic precision, functional correctness, and visual consistency in the generated code. This ensures that Flame generates code that aligns with real-world development standards.

Currently, the framework is designed and optimized for React-based development, taking advantage of its component-based architecture to produce structured, reusable UI components. However, the methodology and pipeline are highly extensible and can be adapted to other front-end frameworks, such as Vue and Angular, with minimal modifications.

This repository provides the full implementation of Flame’s data preparation pipeline, model training process, and evaluation suite, making it an invaluable resource for advancing multimodal front-end code generation research.

Features

- Comprehensive Data Preparation Pipeline: The repository includes scripts and tools for extracting, synthesizing, and structuring multimodal datasets using three distinct data synthesis methods: - Evolution-Based Synthesis - Waterfall-Model-Based Synthesis - Additive Development Synthesis - End-to-End Training Pipeline: Implementation of Flame’s three-stage training strategy, incorporating: - Vision encoder pretraining with public datasets - Image layout interpretation training with synthesized datasets - Full instruction-tuning for image-to-code generation - Evaluation Pipeline for React Code Generation: The repository provides: - The Flame-React-Eval benchmarking dataset - Automated testing scripts for functional correctness and visual fidelity evaluation - Implementation of pass@k evaluation metrics using cosine similarity of rendered outputs Support for Multi-Image Inputs: The model and pipeline enable iterative UI refinement by processing multiple versions of design mockups and updating generated code accordingly.

This repository provides all necessary scripts, models, and evaluation tools to reproduce our experiments and extend Flame for further research in multimodal front-end code generation.