Benchmarks and comparison

What AIR-Core achieves, measured and sourced

We measure the models AIR-Core trains on public data and compare them with well-known references on the same rows. And we compare, with links to each one's documentation, what each tool does when the conversation ends.

Benchmark

Models within the normal range, including where they fall behind

For each dataset, AIR-Core trains the model it would train on the platform: the template's method or, for the others, the one the assistant suggests (random forest). It uses 30% of the rows for testing, shuffled with a fixed seed, and tunes nothing on them. On those same rows we measure the naive prediction and two scikit-learn models with their default values.

Results of the AIR-Core model benchmark, with the naive prediction and two scikit-learn references on the same test rows.
DataTaskTest rowsAIR-Core methodMetricAIR-CoreNaive predictionAIR-Core verdictGradient boosting (scikit-learn)Linear model (scikit-learn)
Customer churn (template)Classification600Logistic regressionCorrect predictions91 %70 %Good87 %91 %
Machine maintenance (template)Classification750Logistic regressionBalanced accuracy78 %50 %Good77 %79 %
Customer feedback (template)Text900Logistic regressionCorrect predictions80 %50 %Good—82 %
Review sentiment: origin (example)Text900Random forestCorrect predictions79 %35 %Good—82 %
Breast cancer (Wisconsin)Classification171Random forestCorrect predictions97 %63 %Good96 %98 %
Palmer penguinsClassification100Random forestCorrect predictions97 %48 %Good99 %99 %
Bank marketingClassification1357Random forestBalanced accuracy60 %50 %Good67 %65 %
Wine quality (red)Regression480Random forestR² (variation explained)45 %0 %Good42 %35 %
AbaloneRegression1254Random forestR² (variation explained)53 %0 %Good52 %53 %

How to read the table

Correct predictions (accuracy) are the share of cases classified correctly; with imbalanced classes we use balanced accuracy, the average of the accuracy for each class. In regression, the share of the variation the model explains (R²). The naive prediction always answers with the most frequent class or the mean.

Where it falls behind

In bank marketing, with highly imbalanced classes, the default random forest falls 5 and 7 points behind the references. On the text datasets and the penguins, 2 or 3 points behind. On the rest, within one point of the best reference, or ahead.

Data and licenses

Templates with synthetic data (CC0), the "Review sentiment" example and five open datasets: breast cancer, bank marketing, wine quality and abalone from the UCI repository (CC BY 4.0), and Palmer penguins (CC0).

Forecasting

The demand template, day by day

AIR-Core tries several methods on the recent past and keeps the most accurate one. Mean error, in units per day, over the last 30 days of that test (lower is better). A gradient boosting model with the calendar is slightly more accurate in that window.

Mean forecast error of the demand template over the last 30 days of its test, in units per day.
Product familyMethod chosen by AIR-CoreAIR-CoreRepeat the previous weekGradient boosting with the calendar (scikit-learn)
BakeryExponential smoothing8107
DrinksExponential smoothing131813
Fresh produceExponential smoothing111310

Reproducible

How we measure it

A program reruns the tests and gives the same figures every time.

  • The same request as the platform. AIR-Core's modeling engine receives what the platform sends when creating a model, and the verdict is calculated by the same quality guarantee code.
  • The same rows for everyone. The program checks that AIR-Core and the references are tested on the same rows and stops if they are not.
  • Nothing tuned on the test. Default values in every method and fixed seeds so that two runs match.
  • Date and versions. 17 November 2026, modeling engine 0.22.0, scikit-learn 1.9.1 and Python 3.15.2, on an Apple M5 Pro.
  • The details, if you ask. The code is in the AIR-Core repository, which is not public. Write to us and we'll send you the method and all the figures. (opens your email app)

Comparison

What keeps working when the conversation ends

BigML trains models very well, and Claude and ChatGPT are excellent for talking with your data. AIR-Core focuses on keeping your team's data, models and decisions alive: connected, monitored, documented and in the EU. Every fact about the other products comes from their documentation or pricing page, with the source number.

Comparison of the capabilities of AIR-Core, BigML, Claude and ChatGPT, with the sources for every fact about the other products.
What you needAIR-CoreBigMLClaudeChatGPT
Connected sources and automatic updatesFiles, URLs, databases, open data (CKAN) and real-time MQTT, with scheduled updates.Sources from URLs, cloud storage and connectors to PostgreSQL, MySQL, SQL Server and Elasticsearch.12 Scheduled refresh of a source: Not verifiedConnectors that read your services when the conversation calls for it; added Google Drive documents stay in sync.1112Connectors that bring in data from your tools; a scheduled task can check a source every weekday.1920
RetrainingAutomatic for classifiers and regressors: on a schedule, when the data changes or when monitoring raises a warning. The new version is only used if it is better.Automatic retraining in BigML Ops, a separate product, or with BigMLer's "retrain" command and your own scheduler.34Not verifiedNot verified
Model monitoring and driftCompares the data each model receives with its training data and measures whether it is still accurate.BigML Ops monitors each model and pairs it with an anomaly detector.3Not verifiedNot verified
Alerts and actionsConditions on your data with email or webhook notifications, and flows that retrain, predict new rows and act.BigML Ops warns you when a model's performance drops below a threshold.3 Notification channel: Not verifiedScheduled tasks (paid plans) that run periodically with your connectors.13 Email or webhook notifications: Not verifiedScheduled tasks and, on supported plans, tasks triggered by Gmail, Slack or GitHub events.21 Email or webhook notifications: Not verified
Quality guaranteeEvery classifier and regressor is compared with the naive prediction. If it does not beat it, AIR-Core tries other methods and, if none does, tells you.Evaluations show the model alongside two references: the most frequent class (or the mean) and a random prediction.5 Automatic warning or retry: Not verifiedNot verifiedNot verified
ExplanationsWeight of each variable in the model and an explanation of each prediction (SHAP values), calculated at training time.Importance of each field in the model and an explanation of each prediction.67Not verifiedNot verified
Model card and AI Act reportCard and trust report for each model (PDF or Markdown) and a register of your organization's AI systems.Summary report for each model, with field importance.8 Documentation for the AI Act: Not verifiedAnthropic publishes cards for its own models (system cards).14 Cards for your models: not applicable.OpenAI publishes cards for its own models (system cards).22 Cards for your models: not applicable.
Hosting in the EU and on your serversCloud on AWS Frankfurt (Germany). Installation on your infrastructure with iCore (Enterprise plan).Its cloud service is hosted in the United States (2022 privacy policy). Private deployment in a public or private cloud, or on-premises.910Data from its products is stored in the United States. Data residency in Europe for Claude models through AWS Bedrock and Google Cloud Vertex.1516 On your servers: Not verifiedWorkspaces with data residency in the United States or the EU for supported content.23 Plans that include it and deployment on your servers: Not verified
MCP connectorOwn MCP server on every plan, the Free one included: connect Claude, ChatGPT or Cursor to your data and models.Not verifiedCustom MCP connectors on every plan; one on the Free plan.17You can add custom MCP servers, subject to your account and workspace policies.24 Plans: Not verified
Entry priceFree plan (€0). Paid plans from €49 a month, excluding VAT.Free plan at $0 a month, with 16 MB per task, and a 3-day trial.10 First paid plan: Not verifiedFree plan at $0. Pro: $17 a month billed annually or $20 billed monthly, excluding tax.18Free plan at $0. Go: $8 a month. Plus: $20 a month.25 Prices in your country: Not verified

"Not verified" means we did not find it in the product's public documentation on 17 November 2026; it does not mean it does not exist. If you spot a mistake, write to us and we'll fix it. (opens your email app)

Limits

What AIR-Core does not do

Not a chat assistant

For chatting, drafting or analyzing a one-off file, Claude or ChatGPT do it very well. AIR-Core connects to them through MCP and keeps what has to keep working afterwards.

Models with default settings

Classical machine learning on tables and text, without deep networks for images. With difficult data, such as the bank marketing above, a specialist can fine-tune further.

No AI Act certification

The card, the trust report and the register help prepare its obligations. They are not legal advice, and your organization decides how each system is classified.

Each plan has its limits

Some features, such as alerts, external databases or the API, depend on the plan, and installation on your servers is a separate project.

Sources

Documentation consulted

Public pages of each product, with the date accessed.

  1. static.bigml.com/static/html-doc/Sources/sect0001.html · 2026-11-17
  2. static.bigml.com/static/html-doc/Sources/sec-data-stores.html · 2026-11-17
  3. bigml.com/bigmlops · 2026-11-17
  4. blog.bigml.com/2018/02/06/retraining-machine-learning-models · 2026-11-17
  5. static.bigml.com/static/html-doc/Classification_and_Regression/sect0046.html · 2026-11-17
  6. static.bigml.com/static/html-doc/Classification_and_Regression/sec-understanding-models.html · 2026-11-17
  7. bigml.com/whatsnew · 2026-11-17
  8. static.bigml.com/static/html-doc/Classification_and_Regression/sect0001.html · 2026-11-17
  9. bigml.com/privacy · 2026-11-17
  10. bigml.com/pricing · 2026-11-17
  11. support.claude.com/en/articles/11176164-use-connectors-to-extend-claude-s-capabilities · 2026-11-17
  12. support.claude.com/en/articles/10166901-use-google-workspace-connectors · 2026-11-17
  13. support.claude.com/en/articles/13854387-schedule-recurring-tasks-in-claude-cowork · 2026-11-17
  14. www.anthropic.com/system-cards · 2026-11-17
  15. privacy.claude.com/en/articles/7996890-where-are-your-servers-located-do-you-host-your-models-on-eu-servers · 2026-11-17
  16. claude.com/regional-compliance · 2026-11-17
  17. support.claude.com/en/articles/11175166-get-started-with-custom-connectors-using-remote-mcp · 2026-11-17
  18. claude.com/pricing · 2026-11-17
  19. chatgpt.com/features/connectors · 2026-11-17
  20. learn.chatgpt.com/use-cases/analyze-data-export · 2026-11-17
  21. learn.chatgpt.com/docs/automations · 2026-11-17
  22. deploymentsafety.openai.com · 2026-11-17
  23. learn.chatgpt.com/docs/enterprise/higher-education-roles-and-permissions · 2026-11-17
  24. developers.openai.com/plugins/deploy/connect-chatgpt · 2026-11-17
  25. learn.chatgpt.com/docs/pricing · 2026-11-17

Start today with your own data

Create your free account and connect your first source in minutes. When you need more, move to a paid plan.