Benchmarks and comparison
What AIR-Core achieves, measured and sourced
We measure the models AIR-Core trains on public data and compare them with well-known references on the same rows. And we compare, with links to each one's documentation, what each tool does when the conversation ends.
Benchmark
Models within the normal range, including where they fall behind
For each dataset, AIR-Core trains the model it would train on the platform: the template's method or, for the others, the one the assistant suggests (random forest). It uses 30% of the rows for testing, shuffled with a fixed seed, and tunes nothing on them. On those same rows we measure the naive prediction and two scikit-learn models with their default values.
| Data | Task | Test rows | AIR-Core method | Metric | AIR-Core | Naive prediction | AIR-Core verdict | Gradient boosting (scikit-learn) | Linear model (scikit-learn) |
|---|---|---|---|---|---|---|---|---|---|
| Customer churn (template) | Classification | 600 | Logistic regression | Correct predictions | 91 % | 70 % | Good | 87 % | 91 % |
| Machine maintenance (template) | Classification | 750 | Logistic regression | Balanced accuracy | 78 % | 50 % | Good | 77 % | 79 % |
| Customer feedback (template) | Text | 900 | Logistic regression | Correct predictions | 80 % | 50 % | Good | — | 82 % |
| Review sentiment: origin (example) | Text | 900 | Random forest | Correct predictions | 79 % | 35 % | Good | — | 82 % |
| Breast cancer (Wisconsin) | Classification | 171 | Random forest | Correct predictions | 97 % | 63 % | Good | 96 % | 98 % |
| Palmer penguins | Classification | 100 | Random forest | Correct predictions | 97 % | 48 % | Good | 99 % | 99 % |
| Bank marketing | Classification | 1357 | Random forest | Balanced accuracy | 60 % | 50 % | Good | 67 % | 65 % |
| Wine quality (red) | Regression | 480 | Random forest | R² (variation explained) | 45 % | 0 % | Good | 42 % | 35 % |
| Abalone | Regression | 1254 | Random forest | R² (variation explained) | 53 % | 0 % | Good | 52 % | 53 % |
How to read the table
Correct predictions (accuracy) are the share of cases classified correctly; with imbalanced classes we use balanced accuracy, the average of the accuracy for each class. In regression, the share of the variation the model explains (R²). The naive prediction always answers with the most frequent class or the mean.
Where it falls behind
In bank marketing, with highly imbalanced classes, the default random forest falls 5 and 7 points behind the references. On the text datasets and the penguins, 2 or 3 points behind. On the rest, within one point of the best reference, or ahead.
Data and licenses
Templates with synthetic data (CC0), the "Review sentiment" example and five open datasets: breast cancer, bank marketing, wine quality and abalone from the UCI repository (CC BY 4.0), and Palmer penguins (CC0).
Forecasting
The demand template, day by day
AIR-Core tries several methods on the recent past and keeps the most accurate one. Mean error, in units per day, over the last 30 days of that test (lower is better). A gradient boosting model with the calendar is slightly more accurate in that window.
| Product family | Method chosen by AIR-Core | AIR-Core | Repeat the previous week | Gradient boosting with the calendar (scikit-learn) |
|---|---|---|---|---|
| Bakery | Exponential smoothing | 8 | 10 | 7 |
| Drinks | Exponential smoothing | 13 | 18 | 13 |
| Fresh produce | Exponential smoothing | 11 | 13 | 10 |
Reproducible
How we measure it
A program reruns the tests and gives the same figures every time.
- The same request as the platform. AIR-Core's modeling engine receives what the platform sends when creating a model, and the verdict is calculated by the same quality guarantee code.
- The same rows for everyone. The program checks that AIR-Core and the references are tested on the same rows and stops if they are not.
- Nothing tuned on the test. Default values in every method and fixed seeds so that two runs match.
- Date and versions. 17 November 2026, modeling engine 0.22.0, scikit-learn 1.9.1 and Python 3.15.2, on an Apple M5 Pro.
- The details, if you ask. The code is in the AIR-Core repository, which is not public. Write to us and we'll send you the method and all the figures. (opens your email app)
Comparison
What keeps working when the conversation ends
BigML trains models very well, and Claude and ChatGPT are excellent for talking with your data. AIR-Core focuses on keeping your team's data, models and decisions alive: connected, monitored, documented and in the EU. Every fact about the other products comes from their documentation or pricing page, with the source number.
| What you need | AIR-Core | BigML | Claude | ChatGPT |
|---|---|---|---|---|
| Connected sources and automatic updates | Files, URLs, databases, open data (CKAN) and real-time MQTT, with scheduled updates. | Sources from URLs, cloud storage and connectors to PostgreSQL, MySQL, SQL Server and Elasticsearch.12 Scheduled refresh of a source: Not verified | Connectors that read your services when the conversation calls for it; added Google Drive documents stay in sync.1112 | Connectors that bring in data from your tools; a scheduled task can check a source every weekday.1920 |
| Retraining | Automatic for classifiers and regressors: on a schedule, when the data changes or when monitoring raises a warning. The new version is only used if it is better. | Automatic retraining in BigML Ops, a separate product, or with BigMLer's "retrain" command and your own scheduler.34 | Not verified | Not verified |
| Model monitoring and drift | Compares the data each model receives with its training data and measures whether it is still accurate. | BigML Ops monitors each model and pairs it with an anomaly detector.3 | Not verified | Not verified |
| Alerts and actions | Conditions on your data with email or webhook notifications, and flows that retrain, predict new rows and act. | BigML Ops warns you when a model's performance drops below a threshold.3 Notification channel: Not verified | Scheduled tasks (paid plans) that run periodically with your connectors.13 Email or webhook notifications: Not verified | Scheduled tasks and, on supported plans, tasks triggered by Gmail, Slack or GitHub events.21 Email or webhook notifications: Not verified |
| Quality guarantee | Every classifier and regressor is compared with the naive prediction. If it does not beat it, AIR-Core tries other methods and, if none does, tells you. | Evaluations show the model alongside two references: the most frequent class (or the mean) and a random prediction.5 Automatic warning or retry: Not verified | Not verified | Not verified |
| Explanations | Weight of each variable in the model and an explanation of each prediction (SHAP values), calculated at training time. | Importance of each field in the model and an explanation of each prediction.67 | Not verified | Not verified |
| Model card and AI Act report | Card and trust report for each model (PDF or Markdown) and a register of your organization's AI systems. | Summary report for each model, with field importance.8 Documentation for the AI Act: Not verified | Anthropic publishes cards for its own models (system cards).14 Cards for your models: not applicable. | OpenAI publishes cards for its own models (system cards).22 Cards for your models: not applicable. |
| Hosting in the EU and on your servers | Cloud on AWS Frankfurt (Germany). Installation on your infrastructure with iCore (Enterprise plan). | Its cloud service is hosted in the United States (2022 privacy policy). Private deployment in a public or private cloud, or on-premises.910 | Data from its products is stored in the United States. Data residency in Europe for Claude models through AWS Bedrock and Google Cloud Vertex.1516 On your servers: Not verified | Workspaces with data residency in the United States or the EU for supported content.23 Plans that include it and deployment on your servers: Not verified |
| MCP connector | Own MCP server on every plan, the Free one included: connect Claude, ChatGPT or Cursor to your data and models. | Not verified | Custom MCP connectors on every plan; one on the Free plan.17 | You can add custom MCP servers, subject to your account and workspace policies.24 Plans: Not verified |
| Entry price | Free plan (€0). Paid plans from €49 a month, excluding VAT. | Free plan at $0 a month, with 16 MB per task, and a 3-day trial.10 First paid plan: Not verified | Free plan at $0. Pro: $17 a month billed annually or $20 billed monthly, excluding tax.18 | Free plan at $0. Go: $8 a month. Plus: $20 a month.25 Prices in your country: Not verified |
"Not verified" means we did not find it in the product's public documentation on 17 November 2026; it does not mean it does not exist. If you spot a mistake, write to us and we'll fix it. (opens your email app)
Limits
What AIR-Core does not do
Not a chat assistant
For chatting, drafting or analyzing a one-off file, Claude or ChatGPT do it very well. AIR-Core connects to them through MCP and keeps what has to keep working afterwards.
Models with default settings
Classical machine learning on tables and text, without deep networks for images. With difficult data, such as the bank marketing above, a specialist can fine-tune further.
No AI Act certification
The card, the trust report and the register help prepare its obligations. They are not legal advice, and your organization decides how each system is classified.
Each plan has its limits
Some features, such as alerts, external databases or the API, depend on the plan, and installation on your servers is a separate project.
Sources
Documentation consulted
Public pages of each product, with the date accessed.
- static.bigml.com/static/html-doc/Sources/sect0001.html · 2026-11-17
- static.bigml.com/static/html-doc/Sources/sec-data-stores.html · 2026-11-17
- bigml.com/bigmlops · 2026-11-17
- blog.bigml.com/2018/02/06/retraining-machine-learning-models · 2026-11-17
- static.bigml.com/static/html-doc/Classification_and_Regression/sect0046.html · 2026-11-17
- static.bigml.com/static/html-doc/Classification_and_Regression/sec-understanding-models.html · 2026-11-17
- bigml.com/whatsnew · 2026-11-17
- static.bigml.com/static/html-doc/Classification_and_Regression/sect0001.html · 2026-11-17
- bigml.com/privacy · 2026-11-17
- bigml.com/pricing · 2026-11-17
- support.claude.com/en/articles/11176164-use-connectors-to-extend-claude-s-capabilities · 2026-11-17
- support.claude.com/en/articles/10166901-use-google-workspace-connectors · 2026-11-17
- support.claude.com/en/articles/13854387-schedule-recurring-tasks-in-claude-cowork · 2026-11-17
- www.anthropic.com/system-cards · 2026-11-17
- privacy.claude.com/en/articles/7996890-where-are-your-servers-located-do-you-host-your-models-on-eu-servers · 2026-11-17
- claude.com/regional-compliance · 2026-11-17
- support.claude.com/en/articles/11175166-get-started-with-custom-connectors-using-remote-mcp · 2026-11-17
- claude.com/pricing · 2026-11-17
- chatgpt.com/features/connectors · 2026-11-17
- learn.chatgpt.com/use-cases/analyze-data-export · 2026-11-17
- learn.chatgpt.com/docs/automations · 2026-11-17
- deploymentsafety.openai.com · 2026-11-17
- learn.chatgpt.com/docs/enterprise/higher-education-roles-and-permissions · 2026-11-17
- developers.openai.com/plugins/deploy/connect-chatgpt · 2026-11-17
- learn.chatgpt.com/docs/pricing · 2026-11-17
Start today with your own data
Create your free account and connect your first source in minutes. When you need more, move to a paid plan.