JetBrains, the company behind a range of popular app development tools, has released its first “open” AI model for coding.
On Wednesday, JetBrains made Mellum, a code-generating model the company released for its various software development suites last year, openly available on the AI dev platform Hugging Face. Mellum, trained on more than 4 trillion tokens, weighs in at 4 billion parameters and is designed specifically for code completion (i.e., completing code snippets based on the surrounding context).
Parameters roughly correspond to a model’s problem-solving skills, while tokens are the raw bits of data that a model processes. A million tokens is equivalent to ~30,000 lines of code.
“Designed for integration into professional developer tooling (e.g., intelligent code suggestions in integrated developer environments), AI-powered coding assistants, and research on code understanding and generation, Mellum is also well-suited for educational applications and fine-tuning experiments,” explains JetBrains in a technical report.
JetBrains says that it trained Mellum, which is Apache 2.0 licensed, on a collection of datasets, including permissively licensed code from GitHub and English-language Wikipedia articles. Training took around 20 days on a cluster of 256 H200 Nvidia GPUs.
Mellum takes some work to get up and running. The base model can’t be used out of the box; it has to be fine-tuned first. While JetBrians has provided a few Mellum models fine-tuned for Python, the company cautions that they’re meant for “estimation about potential capabilities” — not deploying into a production environment.
AI-generated code is no doubt changing how software is built, but it’s also introducing new security challenges. More than 50% of organizations encounter security issues with AI-produced code sometimes or frequently, according to a late 2023 survey by developer security platform Snyk.
Indeed, JetBrains notes that Mellum may “reflect biases present in public codebases” (e.g., generating code similar in style to open source repositories) and that its code suggestions won’t necessarily be “secure or free of vulnerabilities.”
“This is just the beginning,” JetBrains wrote in a blog post. “We’re not chasing generality — we’re building focus. If Mellum sparks even one meaningful experiment, contribution, or collaboration, we would consider it a win.”
Source: https://techcrunch.com/
