Since OpenAI launched ChatGPT, many companies have tried to create their own AI models. However, only a few managed to stand out. Anthropic is one of them.
The AI startup launched its own AI model, Claude, in March. It turned out to be a worthy rival to the GPT-3.5 and GPT-4 models from OpenAI. During this first launch, Anthropic also launched Claude Instant, a lighter, cheaper and faster version of Claude, according to Anthropic. Today, it benefits from an update.
On Wednesday, Anthropic released Claude Instant 1.2, an improved version of the model that builds on Claude 2.0, the latest version of Claude released in July.
Because it uses the advanced capabilities of Claude 2.0, Claude Instant 1.2 has improved significantly in mathematics, coding, reasoning and security, and generates longer and more structured answers, according to the release.
To put the model to the test, Anthropic compared the performance of Claude Instant 1.1 and 1.2 in standard reference evaluations, in particular the Codex evaluation and the evaluation of mathematical problems (GSM8k), which are good references for computing and coding abilities.
In both cases, 1.2 surpassed 1.1 with a score of 58.7% against 52.8% for the original in the Codex evaluation and 86.7% against 80.9% for the original in GSM8k.
For the other reference tests, the new model obtained slightly lower or better results than those of the old model, with minimal differences.
The quality of the answers has also improved, with a decrease in hallucinations and an increased resistance to jailbreaking attempts. An evaluation in red team mode showed that Claude 1.2 is the safest model to use.
Companies can access the new model by filling out an interest form and developers can use the API, which is much cheaper than Claude 2.