lecture 01
Image classification with Create ML
+
Python basics 01

SeAts APp SEAtS ApP SEaTS APP

welcome 👩‍🎤🧑‍🎤👨‍🎤
MLOne (last unit):

- numeric representation
- matrix arithmetic
- layers in neural network
- Apple's Vision framework

and you all made it through!

what this unit is about? {

a combination of theory and practice around AI

a beginning of ChatGPT taking over teaching

the keyword is "training", the becoming of AI

we will have some introduction on
- what training is about,
- what happens at low-level mathematically,
- why we train this way,
- HOW

- what training is about (knowledge🔭)
- what happens at low-level mathematically (knowledge🔭)
- why we train this way (enquiry🤔)
- HOW (process🔧)

This unit will be a combination of theory and practice
(all fun ish, no headaches)

introducing... this unit's menu, to be updated

question time! imagine you are the principle AI engineer at a fast-growing tech startup that is specialised in classifying fruit images being apple🍎 or banana🍌

what skills do you need? what are the steps to take to have such classification AI ? (anything you can think of)

the hidden technical debt in machine learning system

Practice

DATA PREP
- - data collection
- - data pre-processing
TRAINING
- - fine tuning
- - from scratch
DEPLOYMENT
ask chatgpt...

Practice

DATA PREP
- - data collection (p)
- - data pre-processing (p)
TRAINING
- - fine tuning (p,t,c)
- - from scratch (p,t)
DEPLOYMENT (c)
DEPLOYMENT (c)
p: python
t: Turi Create
C: Core ML and Create ML

we are going to...
build Apps with AIs trained by yourself for vision, NLP and audio tasks
we'll also do fun creative ai projects including image generation, audio generation, style transfer, etc.

this lecture:
part 1. training a classifier from scratch for the first time
part 2. python basics

part 1 will be following this typical ML dev pipeline:
- - data collection
- - data pre-processing
- - training in CreateML
- - evaluating and improving

step 1 data collection: fruit images dataset
textbook p213-218

download this starter project , unzip, open ICDemo-working xcodeproj

step 2 data pre-processing:
that's just raw data, often times we need to preprocess the raw data to our needs
this part is usually done by coding in python but CreateML just makes our life easier

summon CreateML 😈

for supervised learning, we need data and their labels.
=>
for image classification, we need images and their classes
=>
How does CreateML know the corresponding label for each image?
ask chatgpt:

Inferred from file structure!
this is also how we should prepare our img classification data (done), when using CreateML
Though when coding in python, there are many other ways to feed in labels information...

step 3 importing dataset into Create ML and training
textbook p220-223

test the model trained with 25 iterations

how to check if a trained model performs well or not?

In CreateML, we can firstly look at the training/evaluation tab,
where the higher accuracy the better

Secondly, thanks to CreateML, we can live preview the results using camera

not working great 🫤
this is a good example of "you should not only looking at the accuracies,
but also inputting images to the model and seeing the classification result yourself!"

so, how to improve?

check out my model with 150 iterations and added noise & flips!

let's bring this to our App
textbook p224, p229-230

python time! meet the black magic...

we will be using
this google colab notebook as 00 of the python series
and
another colab notebook as 01 of the python series

once you open the notebook, go to (upperleft corner) File -> click "Open in playground mode"

clarify on python and colab notebook:
it is analogous to swift(the language) and xcode(the IDE)
but colab has its constraints,
we can switch to use python in another IDE later this semester, and all the python knowledge will remain unchanged

for the first notebook (python basics 00),
read everything from the first cell till "importing packages" (included)

for the second notebook (python basics 01),
read everything from the first cell till the first Exercises,
try to complete Exercise 1. and 2. (homework😬)

done!

in general, python is quite flexible

at some costs of course, but apparantly no data scientist would care😵

ask chatgpt:
a funny python meme on its freedom/flexibility/dynamic typing

homework: find an image classification dataset with a NICE file structure (or build your own)
this one for example
train a classifier and import to App
send a demo