Recipe Generation from Images of Food

An exercise in developing and training models in PyTorch, delving deeper into the field of computer vision and image analysis

As described in the about section, I have strong interests in image processing and computer vision. I believe that there is untapped potential int these fields to assist others day-to-day and improve the overall human condition. To further familiarize myself in the field, I opted to explore imaginative use cases that pushes the boundaries of the field of computer vision. One of these use cases is a model that analyzes an image of a dish and outputs a recipe complete with ingredients, measurements of said ingredients, and instructions to make the dish. That paper is Meta’s Inverse Cooking: Recipe Generation from Food Images (Salvador et al., 2019)

By training the underlying model and comparing its performance to the benchmarks listed in the paper, I was able to do the following:

  • Developed further intuition in computer vision as well as introduce myself to the field of natural language processing.
  • Strengthed my skills in building and training convolutional neural networks and transformer models in PyTorch.
  • Refamiliazed myself with high-performance computing clusters.

References

2019

  1. Inverse Cooking: Recipe Generation from Food Images
    Amaia Salvador, Michal Drozdzal, Xavier Giro-i-Nieto, and 1 more author
    2019