MultiTask Learning for Calorie Prediction
on a Novel LargeScale Recipe Dataset Enriched with Nutritional Information

1. Introduction

Accurate estimation of the caloric content of a dish is a fundamental problem in nutrition informatics, diet planning, and publichealth monitoring. Traditional approaches rely on handcrafted features, ingredientbyingredient summation of nutritional tables, or simple regression models. While these methods can work for wellstructured recipes, they often fail when recipes contain ambiguous ingredient names, cooking procedures that alter nutrient composition, or when the dataset is highly heterogeneous.

Recent advances in deep learning have demonstrated that a single model can be trained to solve several related tasks concurrently a paradigm known as MultiTask Learning (MTL). By sharing representations across tasks, MTL can improve generalisation, especially when some tasks have limited training data. In the context of food analysis, calorie prediction can be coupled with auxiliary tasks such as macronutrient estimation, cuisine classification, and cookingmethod identification.

This page presents an overview of an MTL framework designed for calorie prediction on a newly released largescale recipe dataset that includes detailed nutritional annotations for each entry.

2. The Recipe Dataset

2.1 Scale and Sources

The dataset comprises 1.2million recipes collected from public cooking websites, culinary blogs, and opensource food repositories. Each recipe contains:

• Title and short description
• List of ingredients with quantities (e.g., 2cups of diced tomatoes)
• Stepbystep instructions
• Metadata: cuisine, cooking method, preparation time, and serving size
• Full nutritional panel: calories, protein, fat, carbohydrate, fiber, sugar, sodium, vitamins, and minerals

2.2 Data Cleaning & Normalisation

To make the raw text suitable for neural processing, the following steps were applied:

• Tokenisation of ingredient strings using a custom parser that extracts quantity, unit, ingredient name, and optional descriptors.
• Standardisation of measurement units (e.g., converting tablespoon to ml).
• Mapping of ingredient names to the USDA FoodData Central identifiers for consistent nutrient lookup.
• Removal of duplicate recipes and those missing a complete nutritional panel.

2.3 TrainValidationTest Split

After cleaning, the dataset was split 80%/10%/10% while preserving the distribution of cuisines and cooking methods across splits. The test set contains 120k recipes, providing a robust benchmark for calorie estimation.

Figure 1 Distribution of cuisines and cooking methods in the dataset.

3. MultiTask Learning Architecture

3.1 Overview

The model adopts a shared encoderdecoder design. The encoder processes textual inputs (ingredients and instructions) and creates a dense representation that feeds into several taskspecific heads:

• Calorie Regression Head predicts total calories per serving.
• MacroNutrient Regression Heads predict protein, fat, and carbohydrate values.
• Cuisine Classification Head predicts the cuisine label (e.g., Italian, Chinese).
• CookingMethod Classification Head predicts the primary cooking technique (e.g., baking, frying).

3.2 Encoder Details

The encoder is built on a transformerbased language model pretrained on a large corpus of foodrelated text ( 200M tokens). Two input streams are concatenated:

• Ingredient sequence each ingredient tokenised as [quantity] [unit] [ingredient].
• Instruction sequence raw cooking steps.

The combined sequence is passed through 12 transformer layers (hidden size 768, 12 attention heads). Positional embeddings distinguish ingredient tokens from instruction tokens, encouraging the model to learn their different roles.

3.3 Task Heads

All heads share the same hidden representation h produced by the encoders [CLS] token. The heads are simple feedforward networks:

CalorieHead(x) = ReLU(Linear(x, 256))  Linear(256, 1)MacroHead(x)   = ReLU(Linear(x, 256))  Linear(256, 3)   // protein, fat, carbsCuisineHead(x) = ReLU(Linear(x, 128))  Linear(128, C)   // C = #cuisinesMethodHead(x)  = ReLU(Linear(x, 128))  Linear(128, M)   // M = #methods

3.4 Loss Function

The overall loss is a weighted sum of individual task losses:

L = L_calorie + L_macro + L_cuisine + L_method

where:

• L_calorie = Mean Squared Error (MSE) between predicted and true calories.
• L_macro = MSE across the three macronutrients.
• L_cuisine and L_method = Crossentropy losses.

Hyperparameters were tuned on the validation set; the final configuration gave the calorie task the highest weight (=2.0) while still leveraging auxiliary signals.

4. Training Procedure

4.1 Optimisation

Training used AdamW with =0.9, =0.999, weight decay=0.01, and a cosine learningrate schedule with warmup for the first 5% of steps. The batch size was 256, and the model converged after roughly 12epochs ( 1.5M steps).

4.2 Data Augmentation

To improve robustness:

• Ingredient quantities were randomly perturbed by 10%.
• Synonym replacement for ingredient names (e.g., bell pepper capsicum).
• Instruction shuffling within a small window to mimic varied writing styles.

4.3 Evaluation Metrics

Primary metric for calorie prediction is Mean Absolute Error (MAE) expressed in kilocalories (kcal) per serving. Secondary metrics include:

• Root Mean Squared Error (RMSE) for macronutrients.
• Top1 accuracy for cuisine and cookingmethod classification.

5. Results

5.1 Calorie Prediction

On the heldout test set, the multitask model achieved:

• MAE = 23.5kcal ( 4.2% relative error for an average 560kcal dish).
• RMSE = 31.8kcal.

For comparison, a singletask baseline (identical encoder, only calorie head) yielded MAE=31.7kcal, highlighting a 25% improvement thanks to auxiliary tasks.

5.2 Auxiliary Tasks

• Macronutrient RMSE: protein=2.8g, fat=3.1g, carbs=4.5g.
• Cuisine classification accuracy: 82% (46 classes).
• Cookingmethod classification accuracy: 88% (12 methods).

5.3 Ablation Study

Removing the cuisine head increased calorie MAE to 26.9kcal, while removing the method head raised it to 27.4kcal. Excluding macronutrient heads caused the largest degradation (MAE=30.1kcal), confirming that regression of related nutrients is the most beneficial auxiliary signal.

Figure 2 MAE of calorie prediction across different model configurations.

6. Discussion

The experiments demonstrate that multitask learning can substantially improve calorie estimation on noisy, realworld recipe data. Sharing a common representation enables the model to capture latent food semantics such as ingredients typical of a highfat cuisine or cooking methods that increase energy density. Moreover, the auxiliary classification tasks act as regularisers, preventing overfitting to the calorie regression target.

Several practical implications arise:

• Scalability One model simultaneously provides several nutritional outputs, reducing the need for separate pipelines.
• Explainability Attention visualisation shows which ingredients and steps contribute most to the predicted calories.
• Personalisation The same architecture can be finetuned for specific dietary regimes (e.g., lowsodium, keto) by adding taskspecific heads.

Limitations include reliance on the accuracy of the underlying ingredienttonutrient mapping and the fact that cooking transformations (e.g., oil absorption) are only implicitly learned from data. Future work could integrate physicsbased nutrientloss models or exploit multimodal inputs such as food images.

7. Conclusion

We introduced a comprehensive multitask learning framework for calorie prediction on a novel, largescale recipe dataset enriched with detailed nutritional information. By jointly learning calorie regression, macronutrient estimation, cuisine, and cookingmethod classification, the model achieves stateoftheart accuracy while remaining compact and versatile. The public release of the dataset and codebase is expected to accelerate research at the intersection of nutrition science and machine learning.