AI FOUNDATIONS / BEGIN HERE
Before a library, a model, or a training loop, understand the ideas behind them. Follow AI from its early ambitions to the systems and tools you can use today.
Its history brings together logic, search, statistics, learning, and computing. These selected milestones overlap; one school of thought did not simply erase the others.
Alan Turing
Computing Machinery and Intelligence introduces the imitation game: evaluate a machine through its behavior in conversation. This frames a research question; it does not establish that fluent behavior proves consciousness.
What to take forward: Define observable performance before making a claim about intelligence.
Turing’s original paper ↗The Dartmouth research project
The Dartmouth summer project helps establish artificial intelligence as a research field. Its ambition is much wider than chatbots: reasoning, learning, language, and solving problems with machines.
What to take forward: AI is the broad field. Any one technique covers only part of it.
Dartmouth’s historical account ↗Frank Rosenblatt and the perceptron
Rosenblatt demonstrates a system that adjusts connections to distinguish input patterns. Early learning machines make the central ML idea concrete: useful behavior can come from adapting parameters to examples.
What to take forward: Learning is not new, and a simple learned model is not a modern deep network.
Cornell’s perceptron history ↗The Lighthill controversy
Criticism of AI’s progress and its ability to scale contributed to a difficult period for British AI research. The history of AI includes setbacks and periods known as AI winters, alongside continued work.
What to take forward: Success on a demonstration does not establish that a method scales to the real world.
University of Edinburgh’s AI history ↗MYCIN and expert systems
Expert systems encode domain knowledge as rules and use an inference engine to apply it. MYCIN becomes a major research example. Building and maintaining the knowledge base is a central engineering challenge.
What to take forward: An AI system can use human-authored knowledge without learning its rules from a dataset.
Buchanan & Shortliffe’s MYCIN book ↗Rumelhart, Hinton, and Williams
An influential paper shows how backpropagation trains hidden units to represent useful features. Backpropagation computes gradients; an optimization procedure uses them to adjust weights.
What to take forward: A network can learn both intermediate features and a final prediction.
The backpropagation paper ↗IBM Deep Blue
Deep Blue defeats world chess champion Garry Kasparov in a match. Specialized search, evaluation, and computing power produce extraordinary performance within a well-defined domain.
What to take forward: Being excellent at chess does not imply competence at every other task.
IBM’s Deep Blue history ↗Krizhevsky, Sutskever, and Hinton
The AlexNet work demonstrates strong image-classification results using a deep convolutional network. Large labeled datasets, GPU computation, and training techniques make a powerful combination.
What to take forward: Progress depends on data and computation as well as the model’s architecture.
The ImageNet / AlexNet paper ↗DeepMind’s AlphaGo
AlphaGo defeats Lee Sedol in Go. Its approach combines neural networks, learning, and search, illustrating how different AI methods can work together in one system.
What to take forward: The choice is often a combination of methods, rather than one technique replacing all others.
DeepMind’s AlphaGo account ↗Vaswani and collaborators
Attention Is All You Need introduces the Transformer for sequence transduction. Attention lets representations incorporate information from other positions; Transformers become foundational to many later language and multimodal models.
What to take forward: An architecture is a design for computation, not a complete application.
The Transformer paper ↗The launch of ChatGPT
OpenAI releases ChatGPT on November 30, 2022. A conversational interface makes a trained language model directly accessible to a broad audience, while also exposing its tendency to produce plausible errors.
What to take forward: A useful interface can change adoption without making the underlying model infallible.
The original launch announcement ↗An expanding and uneven landscape
Stanford’s 2026 AI Index documents progress across language, vision, reasoning, and agentic tasks, alongside uneven reliability. The practical question becomes how well a complete system performs in its intended setting.
What to take forward: Evaluate the task, data, tools, and operating conditions together.
Stanford AI Index 2026 ↗Artificial intelligence studies and builds computational systems that perform tasks involving perception, reasoning, language, planning, learning, or action. A system’s goal and environment determine what “good” behavior means.
Recognize speech, identify objects, or extract text from a page. The input can be sound, images, text, sensor readings, or a mixture.
Search routes, satisfy scheduling constraints, or select the next move in a game. Rules and search can work alongside learned models.
Estimate demand, classify a ticket, recommend an item, or generate a draft. Prediction quality depends on the task and the evidence available.
A robot is a physical system that may use AI, but AI does not require a body. Automation executes a process; it may use fixed software, AI, or both. A language model is a model; a chat application also includes interfaces, tools, policies, and other software.
Further grounding: NIST’s framework for AI systems and their context.
A rule-based system starts with knowledge expressed by people. A machine-learning system uses a learning procedure to fit a model from examples or experience. Both still depend on human choices about the task, representation, and evaluation.
Generative AI overlaps this map as a capability. Modern generators commonly use deep learning.
The broad field of building systems that perceive, reason, plan, learn, or act toward objectives.
Example: a route planner searches possible paths using a map and a cost function. Its search can work without a learned model.
| Approach | Where behavior comes from | Example | Key question |
|---|---|---|---|
| Rules and search | Explicit knowledge, possible actions, and a search or inference procedure | A planner finds a route satisfying constraints. | Are the rules, state, and objective adequate? |
| Classical ML | Fitting a model using data and an objective | A decision tree predicts a ticket category. | Does it generalize to appropriate held-out examples? |
| Deep learning | Training a multilayer neural network | A network recognizes patterns in images. | Do the data, architecture, and training support the task? |
| Hybrid system | A combination of learned and explicitly programmed components | A model classifies a request; rules route it; search retrieves a document. | Does the complete workflow succeed, including its failure paths? |
A toy support desk assigns priority using only waiting time. The hand-written rule uses 8 hours. The learned classifier tries seven candidate thresholds and keeps the one with the fewest training errors.
2 / 6 training examples disagree with this rule.
0 / 6 training errors. Changing a label refits the threshold; moving the new input only runs inference.
This is a real, tiny threshold search in your browser. The examples are invented. Zero training errors say nothing yet about unseen tickets; real priority may depend on information this toy model omits.
Read more: Google’s introduction to machine learning. These learning settings describe how supervision is supplied; they are not separate brands of AI software.
There is no universal AI mechanism. Here is the lifecycle of a learned prediction system, using support-ticket classification as a running example.
Input: the ticket available when it arrives. Output: a category or priority. Decide which mistakes matter and when the system should defer to a person.
Gather representative examples, inspect labels, handle missing data, and convert inputs into a representation the model can use. Future information must not sneak into the inputs.
Use training data to fit the model, validation data to choose settings, and an appropriate reserved test set to assess the final procedure. Respect groups and time when a random split would be misleading.
Begin with a simple reference. A training algorithm adjusts parameters to improve an objective. For many neural models, backpropagation computes gradients and an optimizer updates weights.
Inspect false positives and negatives, performance across relevant groups, unusual inputs, latency, and cost. A small training error can coexist with poor real-world predictions.
Apply the fitted model to a new ticket, route the result, and measure behavior over time. Changes in customers, language, or labeling can reduce performance and prompt a new development cycle.
Twenty-four support tickets arrive in order. Eight customers appear across them, several more than once. Choose how to reserve evaluation rows and watch two specific problems appear and disappear.
Fits the model · Held out · C# is the customer on that ticket.
Every row is equally likely to be held out. This is the usual default, and it is the one that quietly breaks when rows share a customer or arrive in a meaningful order.
Both counts above are leakage you can measure before you look at any score. A customer on both sides lets the model recognise a customer rather than the problem. Training on rows that arrived after the evaluation period lets it use information the real system would not have had. Either one inflates the result without changing a single line of the model.
No. It is the right default when rows are independent and order carries no meaning — separate patients in a one-off study, say. It goes wrong exactly when a hidden connection survives the shuffle. Ask what a row shares with other rows, and what the system will know at the moment it has to predict. Answer those two questions and the split usually chooses itself.
A learning procedure changes the model. Some methods optimize differentiable losses; others fit trees or use different procedures. Training is not always gradient descent.
The fitted model produces a prediction or generated output. An ordinary request does not necessarily update its weights, even when the surrounding application saves conversation history.
Implementation connection: scikit-learn on preprocessing, leakage, and pipelines.
For a typical autoregressive language model, generation is a sequence of computations over the available context. The chat interface is only the visible end of that system.
Text becomes tokens, which may be words, word pieces, or other units. The system turns those tokens into numeric representations.
In a Transformer, attention helps representations incorporate information from other positions. The trained network computes scores for possible next tokens.
A decoding procedure chooses a token from those scores. That token joins the context, and the computation continues until a stopping condition is met.
The surrounding system may retrieve documents, run code, or call an authorized tool. Its result can become new context for another model step.
Sources: Google’s language-model introduction and the Transformer paper.
A tokenizer cuts text into the units a model has representations for. Type anything and watch where the cuts land. A ## marks a piece that continues the word before it.
This vocabulary has 116 hand-picked entries. A real one is learned from a corpus — BPE, WordPiece, or Unigram — and runs to tens of thousands of entries covering every byte, so nothing is ever unreadable. What carries over is the cost: a common word is one token, a rare or misspelt one shatters into several. Context limits, pricing, and the meaning of “a 1,000-token document” are all counted in these units, not in words.
Ask a model how many letter R’s are in a word and it is not looking at letters — it is looking at a handful of pieces it has representations for. The spelling has to be reconstructed from those pieces rather than read off. It is a good reminder that a model’s inputs are not the same objects you see on screen, and that a failure can come from the representation rather than from reasoning.
Context: “The sky is …” Imagine three candidate tokens with fixed scores. Change temperature to see how their relative probabilities change.
These scores and single-word candidates are illustrative, not output from a trained LLM. Lower temperature concentrates the distribution; it does not verify facts. Real vocabularies include subwords, and sampling settings depend on the model.
Once a piece of text is a vector, “related” becomes arithmetic. Pick a word, then compare the two measures that answer “related” differently.
NEAREST TO “CAT”
Furthest away: keyboard (-0.992).
Switch measures with cat selected. Cosine puts kitten first, because it points in almost the same direction. Euclidean distance puts dog first, because kitten’s vector is shorter and that gap counts against it. Retrieval systems usually normalise vectors and use cosine for exactly this reason: they want the topic, not the magnitude.
These thirteen coordinates were placed by hand so the geometry is visible; nothing here was learned. A real embedding has hundreds or thousands of dimensions, no individual axis means anything you could name, and the neighbourhoods come from training data — along with whatever associations that data happened to contain. The mechanism is the same one behind semantic search and the retrieval step in RAG: embed the question, then find the stored vectors nearest to it.
It means the text is similar to the query in the embedding space, which is not the same as containing the answer. Retrieval can return a passage that reads like the question and says nothing useful, and it can miss the one passage that matters because the wording diverges. That is why the retrieval step gets evaluated separately from the generated answer: first ask whether the right passage came back, then ask whether the answer actually follows from it.
A foundation model is trained broadly enough to support a range of downstream uses. Pretraining learns patterns from large datasets. Additional training may improve instruction following, specialized behavior, or alignment with feedback. The final application adds context, tools, interfaces, and evaluations.
Change the instructions, examples, or context for a request. You are usually conditioning a model with fixed weights.
Find relevant external material and provide it to the generator. This changes the evidence available at answer time; it does not guarantee the answer uses that evidence correctly.
RAG paper ↗Update parameters through further training on selected examples. It can adapt behavior, but it needs suitable data and evaluation. It is not a substitute for a reliable source of changing facts.
Text generation is one example. Diffusion models learn a denoising process and can generate samples through repeated refinement. Generating an image does not have to mean predicting the next text token. Denoising diffusion research ↗
This snapshot was checked on September 19, 2026. It describes documented directions and available tools, rather than predicting when a future capability will arrive.
Stanford’s 2026 AI Index reports that 88% of surveyed organizations used AI in 2025, while agent adoption was still at an earlier stage. That is a survey result about organizations in the study, not a claim that 88% of every business task is automated. Read the economic evidence ↗
Text, images, audio, and video increasingly share one interface. This enables tasks such as asking about a diagram or discussing a document by voice. Supported inputs and outputs vary by product and model.
What it means for you: Check the modality you actually need, not just the model’s headline label.
Gemini’s current capabilities ↗Systems can combine a model with tools, state, and a loop that chooses subsequent actions. This expands the task from producing text to navigating a workflow, where tool failures and incorrect actions also matter.
What it means for you: Measure task completion and inspect actions; a fluent explanation is not proof of execution.
Anthropic: workflows and agents ↗The 2026 AI Index reports stronger performance on demanding benchmarks alongside weaknesses on seemingly simpler tasks. Benchmark success does not establish dependable performance on every input or a particular workplace task.
What it means for you: Use your own representative evaluation set, including exceptions and failure cases.
Stanford AI Index 2026 ↗Retrieval-augmented generation supplies relevant documents at answer time. It offers a way to use changing or specialized information without fitting all those facts into a fresh model update.
What it means for you: Evaluate whether retrieval found the right passage and whether the answer actually follows it.
The RAG research paper ↗Model hubs and local runtimes make it possible to explore downloadable models as well as hosted services. Access to weights, hardware needs, and license terms differ; downloadable does not automatically mean unrestricted.
What it means for you: Compare capability, memory, latency, operational effort, and the actual license.
Hugging Face Hub documentation ↗The NIST AI Risk Management Framework treats evaluation and risk management as ongoing work across the system lifecycle. Reliability, bias, and the context of use need attention beyond the initial demonstration.
What it means for you: Monitor changes after deployment and define when a person should take over.
NIST AI Risk Management Framework ↗An assistant, a model, a library, and a cloud platform solve different parts of a problem. This representative guide covers 16 tools; it is not a ranking or an exhaustive market directory.
| You want to… | Start by exploring… | Compare using… |
|---|---|---|
| Use AI for everyday work | An assistant or research application | Result quality, source support, document handling, and account controls |
| Predict an outcome from a table | Python data tools and a classical ML library | A baseline, leakage-free validation, and the errors that matter |
| Build a neural model | A deep learning framework and appropriate data | Task performance, training effort, hardware, and deployment needs |
| Add an AI feature to an application | A model API or local runtime, plus application code | Latency, cost, data handling, reliability, and integration effort |
| Coordinate actions across tools | A workflow or agent framework | Actual task completion, recovery behavior, permissions, and monitoring |
Product names and capabilities were checked against official pages on September 19, 2026. Plans, prices, regional availability, and model catalogs can change; use each official link for current details.
16 of 16 tools shown
Hosted application
Conversational help with writing, explanation, analysis, and tasks involving supported tools.
When comparing: Check enabled tools, account settings, and evidence behind factual answers.
Official product / documentation ↗Hosted application
Work with language, documents, analysis, and coding through a conversational assistant.
When comparing: Check document limits, connected tools, and the behavior required by your task.
Official product / documentation ↗Hosted application
A Google assistant for questions and tasks across supported text and multimodal inputs.
When comparing: Check which integrations and input types are available in your account and region.
Official product / documentation ↗Search and answer application
Find information through web-backed answers and source links.
When comparing: Open the cited page and verify that it supports the specific claim.
Official product / documentation ↗Coding assistant and agent
Code suggestions, explanations, edits, and agent workflows within supported development environments.
When comparing: Review diffs, run tests, and verify repository and tool permissions.
Official product / documentation ↗AI coding environment
Explore a codebase and make changes through an editor with AI assistance and agent features.
When comparing: Check the scope of proposed edits and validate the resulting behavior.
Official product / documentation ↗Generative creative application
Generate and edit visual assets using supported image and video workflows.
When comparing: Review output quality, usage terms, and the requirements of your intended project.
Official product / documentation ↗Generative media platform
Create and transform media, including video, with generative models and editing tools.
When comparing: Check continuity, controllability, export options, and applicable usage terms.
Official product / documentation ↗Python ML library
Fit classical models, compose preprocessing pipelines, and perform model selection and evaluation.
When comparing: Keep preprocessing inside training folds and choose metrics suited to the task.
Official product / documentation ↗Deep learning framework
Build tensor computations and neural networks with automatic differentiation.
When comparing: Check device support, tensor shapes, training cost, and reproducibility.
Official product / documentation ↗Deep learning framework and API
Define, train, evaluate, and deploy neural models; used in ML Week 4.
When comparing: Match the output, labels, and loss, and save the required preprocessing.
Official product / documentation ↗Model and dataset hub
Discover model weights, datasets, model cards, and demonstrations.
When comparing: Inspect the model card, data provenance, license, and task-specific limitations.
Official product / documentation ↗Model runtime
Run supported models through a local interface or supported cloud features.
When comparing: Local models still require suitable memory and compute. Check whether your chosen route is local or remote.
Official product / documentation ↗Workflow and agent framework
Coordinate stateful model-and-tool workflows with control over execution.
When comparing: You still need a model, tools, evaluation, and a deployment plan.
Official product / documentation ↗Managed cloud platform
Access supported foundation models and build generative AI applications and agents on AWS.
When comparing: Check model availability, region, data handling, and usage costs.
Official product / documentation ↗Managed cloud platform · formerly Vertex AI
Google Cloud’s platform for building and operating AI models, applications, and agents.
When comparing: Check supported services, region, evaluation options, and operating costs.
Official product / documentation ↗Choose the test before choosing the tool. “Looks intelligent” is not a success criterion.
A generated explanation can contain invented facts or citations. Check the referenced material and recompute consequential calculations.
A model may work on common cases and fail on a different group, time period, language, or environment. Test the conditions that match intended use.
Retrieval can miss evidence, a tool call can fail, and an action can be inappropriate. Inspect the entire workflow, including access to data and the point at which a person reviews a result.
Lifecycle perspective: NIST AI Risk Management Framework.
You are ready to move on when you can describe a system without treating every part as a mysterious “AI.”
Yes. Search, planning, logic, and human-authored expert rules can produce useful AI behavior without fitting a model from examples. Many real systems combine these with learned components.
AI is the broad field. ML is an approach within it that learns from data or experience. Deep learning is a family of ML methods based on multilayer neural networks. Generative AI describes a capability: creating outputs such as text or images. Modern generative systems often use deep learning; an LLM is one kind of model, not a synonym for all AI.
Training adjusts learned parameters according to data and an objective. Inference applies a fitted model to an input. An ordinary request can change context or application state without changing model weights.
No. Fluency and confidence in wording do not establish correctness. Check source support, calculations, and task results separately.
Prompting changes the instructions or context provided for a request. Retrieval selects external information and adds it to that context. Fine-tuning updates model parameters using additional training examples. They address different needs and can be combined.
An agentic system can choose actions, call tools, observe results, and continue toward an objective. The surrounding software controls permissions, state, and execution. A model claiming that an action happened is not enough; the tool result must confirm it.
It makes improvement measurable. A fixed rule, majority-class predictor, or simple model may already solve enough of the task. Complexity needs to earn its additional cost and maintenance.
Define the task and expected output, representative inputs, acceptable errors, data constraints, and a time or cost budget. Then compare actual results using the same examples.
Python libraries supply operations for arrays, tables, plots, and models. ML supplies the discipline for learning from data and evaluating what was learned. Knowing one does not replace the other.
Already comfortable with Python? Start ML Week 0. Need the language first? Review Python fundamentals. Deep learning follows in the DL learning map.