کتاب هوش مصنوعی برای مهندسان (۲۰۲۵)

Preface
Acknowledgments
Contents
۱ Preparation Knowledge: Basics of AI
۱.۱ Overview
۱.۲ Introduction to Artificial Intelligence
۱.۲.۱ Why Look into AI?
۱.۲.۲ What Is AI?
۱.۲.۳ History of AI
Spawning (1930–۱۹۵۲)
Birth (1952 and 1956)
Symbolic AI (1956–۱۹۷۴)
First AI Winter (1974–۱۹۸۰)
Expert System and Connectionism Bloom (1980–۱۹۸۷)
Second AI Winter (1987–۱۹۹۳)
Recovery (1993–۲۰۱۱)
Deep Learning and Big Data Rise (2011–present)
۱.۲.۴ AI Versus Traditional Engineering Methods
Practice: Prediction of Object Flying Trajectory (Physics Methods Versus Data Method)
۱.۲.۵ AI Applications
AI Applications in All Sectors
AI Applications in Engineering
۱.۳ Basics of AI
۱.۳.۱ Basic Concepts
Key Machine Learning Elements
Data Format
Machine Learning Workflow
۱.۳.۲ Common Algorithms
Overview and Machine Learning Tasks
Supervised Learning
Unsupervised Learning
Reinforcement Learning
Semi-supervised Learning
Summary
۱.۳.۳ Challenges and Issues in Machine Learning
Data Issues
Inductive Bias
Underfitting and Overfitting
۱.۴ Practice: Gain First Experience with AI via a Machine Learning Task
۲ Tools for Artificial Intelligence
۲.۱ Overview of Tools for AI
۲.۲ Python
۲.۲.۱ Introduction to Python Coding Environment
۲.۲.۲ Basics
۲.۲.۳ Variables and Data Types
۲.۲.۴ Operators
۲.۲.۵ Conditional Control Statements
۲.۲.۶ Sequential Control Statements
۲.۲.۷ Functions
۲.۲.۸ Input and Output
۲.۲.۹ Advanced Python Functionality
۲.۳ Data Manipulation and Visualization
۲.۳.۱ NumPy
NumPy Array
Array Constructions
Array Operations
۲.۳.۲ Pandas
From NumPy to Pandas
Series
Dataframe
۲.۳.۳ Matplotlib
Pyplot: Procedural Plotting Interface
Object-Oriented Plotting Interface
۲.۴ General Machine Learning
۲.۴.۱ Scikit-Learn
Data Import
Data Preprocessing
Using Models
Saving Models
۲.۵ Deep Learning
۲.۵.۱ Deep Learning Frameworks
۲.۵.۲ TensorFlow
Overview of APIs
Computational Graph
Variables
Placeholders and Comprehensive Example
Comprehensive Example
۲.۵.۳ Keras
Installation and Data Preparation
Model Establishment with the Sequential API
Model Establishment with the Functional API
Training and Result Visualization
۲.۶ Reinforcement Learning
۲.۶.۱ Overview of RL Tools
۲.۶.۲ OpenAI Gym
۲.۷ Practice: Use, Compare, and Understand TensorFlow and Keras for Problem-Solving
۳ Linear Models
۳.۱ Overview
۳.۲ Basics of Linear Models
۳.۲.۱ Simple Explanation of Linear Models
۳.۲.۲ General Formulation of Basic Linear Model
۳.۳ Other Linear Regression Algorithms
۳.۳.۱ Ridge
۳.۳.۲ Lasso
۳.۴ Logistic Regression for Classification
۳.۴.۱ Binary Classification
۳.۴.۲ Multiclass Classification
۳.۵ Making Linear Models Nonlinear via Kernel Functions
۳.۵.۱ Mapping Data to Higher-Dimensional Space with Stretching Functions
۳.۵.۲ Kernel Functions
۳.۶ Practice: Develop Code to Implement the Basic Linear Model
۴ Decision Trees
۴.۱ Overview
۴.۲ Basics of Decision Trees
۴.۳ Classic Decision Tree Algorithms
۴.۳.۱ ID3 Algorithm
۴.۳.۲ C4.5 Algorithm
۴.۳.۳ CART Algorithm
۴.۳.۴ Implementation
۴.۴ Issues and Techniques: Overfitting and Pruning
۴.۴.۱ Pre-Pruning
۴.۴.۲ Post-Pruning
Cost-Complexity Pruning (CCP)
Reduced Error Pruning (REP)
Pessimistic Error Pruning (PEP)
Minimum Error Pruning (MEP)
Comparison and Summary
۴.۵ Practice: Decision Trees in Scikit-learn—Training, Tree Plot, and Testing
۵ Support Vector Machines
۵.۱ Overview
۵.۲ Basics of SVM: Hard-Margin SVM
۵.۲.۱ Basic Formulation
۵.۲.۲ Dual Formulation
۵.۳ Generalization of SVM: Kernel Methods
۵.۴ Soft-Margin SVM
۵.۴.۱ Basic Formulation
۵.۴.۲ Dual Formulation
۵.۵ More About SVM
۵.۵.۱ SMO Algorithm
۵.۵.۲ SVM for Multiclass Classification and Regression
۵.۶ Practice: Use of SVMs in Scikit-Learn for Classification and Regression
۶ Bayesian Algorithms
۶.۱ Overview
۶.۲ Statistics Background for Machine Learning
۶.۲.۱ Statistics and Machine Learning
۶.۲.۲ Frequentists and Bayesians
۶.۲.۳ Overview of Statistical Inference
۶.۲.۴ Maximum Likelihood Estimation (MLE)
۶.۲.۵ Bayesian Estimation
۶.۳ Parametric Bayesian Methods
۶.۳.۱ Naive Bayes Classifier
۶.۳.۲ Semi-Naive Bayesian Classifier
One-Dependent Estimator (ODE)
Variations of ODE
Tree Augmented Naive Bayes (TAN)
۶.۳.۳ Bayesian Network
Structure
Implementation
۶.۴ Bayesian Nonparametrics
۶.۴.۱ Parametric Versus Nonparametric Models
Overview
Parametric Models
Nonparametric Models
From Parametric to Nonparametric Bayesian Algorithms
۶.۴.۲ Gaussian Processes
Introduction to Gaussian Process
Modeling Functions Using Multivariate Gaussian
Making Predictions Using a Prior and Observations
Example
Summary
۶.۵ Practice: Code Gaussian Naive Bayes Classifier, Try Bayesian Network, and Apply Gaussian Process
۷ Artificial Neural Networks
۷.۱ Overview
۷.۲ Basics of Artificial Neural Networks
۷.۲.۱ From Biological Neural Network to ANN
۷.۲.۲ Activation Function
۷.۲.۳ Perceptron
۷.۲.۴ Multiple-Layer Feedforward Neural Network
۷.۳ Training with Backpropagation
۷.۳.۱ Concepts
۷.۳.۲ Backpropagation in a 3-Layer Network
۷.۳.۳ Backpropagation in Neural Networks with 3+ Layers
۷.۴ Implementation
۷.۴.۱ Practical Skills
۷.۴.۲ Procedure for An Example
۷.۴.۳ *Shape and Arrangement of Arrays for Data
۷.۵ Other ANN Issues
۷.۶ Practice: Modify and Assess the Architecture of an ANN
۸ Deep Learning
۸.۱ From Artificial Neural Networks to Deep Learning
۸.۱.۱ Overview
۸.۱.۲ The First Wave
۸.۱.۳ The Second Wave
۸.۱.۴ The Third Wave
۸.۱.۵ Summary of Enabling Innovations
۸.۲ Convolutional Neural Network
۸.۲.۱ Convolution
Forward Pass
Backward Pass
Padding and Stride
۸.۲.۲ ReLU
۸.۲.۳ Pooling
۸.۳ Recurrent Neural Network
۸.۳.۱ Forward Pass
۸.۳.۲ Backward Pass
۸.۴ Practical Deep Learning Skills
۸.۴.۱ Initialization
Overview
Xavier Initialization
He Initialization
LeCun Initialization
Batch Normalization
۸.۴.۲ Optimization Methods
SGD
Momentum
Nesterov
AdaGrad
AdaDelta
RMSprop
Adam
Nadam
۸.۴.۳ Data Preprocessing and Augmentation
۸.۵ Practice: Build AlexNet Using Keras to Address MNIST Image Classification
۹ Ensemble Learning
۹.۱ Overview
۹.۲ Basics of Ensemble Learning
۹.۲.۱ Definition
۹.۲.۲ Basic Questions
۹.۲.۳ Categories of Ensemble Learning Methods
۹.۲.۴ Essence of Ensemble Learning
۹.۲.۵ History and Challenge
۹.۳ Bagging
۹.۳.۱ Basic Bagging
۹.۳.۲ Random Forest
۹.۴ Boosting
۹.۴.۱ AdaBoost
Loss Function
Update on Model Weights
Update on Sample Weights/Distribution
Pseudo-Code
۹.۴.۲ Gradient Boosting
۹.۵ Stacking
۹.۶ Practice: Code and Evaluate Ensemble Learning Methods
۱۰ Clustering
۱۰.۱ Overview
۱۰.۲ Basics of Unsupervised Learning
۱۰.۲.۱ From Supervised Learning to Unsupervised Learning
۱۰.۲.۲ Framework for Unsupervised Learning
۱۰.۲.۳ Overview of Clustering
۱۰.۳ K-Means Clustering
۱۰.۳.۱ Math Framework of K-Means Algorithm
۱۰.۳.۲ Implementation of K-Means
۱۰.۳.۳ Initialization
۱۰.۳.۴ Selection of K
۱۰.۳.۵ Pros and Cons
۱۰.۴ Mean-Shift Clustering Algorithm
۱۰.۴.۱ Pros and Cons
۱۰.۵ Density-Based Spatial Clustering (DBScan)
۱۰.۵.۱ Pros and Cons
۱۰.۶ Gaussian Mixture Models (GMM)
۱۰.۶.۱ Pros and Cons
۱۰.۷ Hierarchical Agglomerative Clustering (HAC)
۱۰.۷.۱ Pros and Cons
۱۰.۸ Evaluation of Clustering
۱۰.۸.۱ Overview of Evaluation Metrics
۱۰.۸.۲ Internal Evaluation
Silhouette Coefficient
Davies-Bouldin Index
Dunn Index
۱۰.۸.۳ External Evaluation
Rand Index
Adjusted Rand Index
Normalized Mutual Information (NMI)
Fowlkes-Mallows Index
Contingency Matrix
۱۰.۹ Practice: Test and Modify Clustering Code for Problem-Solving
۱۱ Dimension Reduction
۱۱.۱ Overview
۱۱.۲ Basics of Dimension Reduction
۱۱.۲.۱ Concepts and Needs
۱۱.۲.۲ Popular Methods and Classification
۱۱.۳ Common Feature Selection Methods
۱۱.۴ Feature Extraction Method 1: Principal Component Analysis
۱۱.۴.۱ Concept and Main Idea
۱۱.۴.۲ Theoretical Basis
Deduction Based on Minimum Distance
Deduction Based on Maximum Variance
۱۱.۴.۳ Implementation
۱۱.۵ Feature Extraction Method 2: Linear Discriminant Analysis
۱۱.۵.۱ Concept and Main Idea
۱۱.۵.۲ Theoretical Basis
Rayleigh Quotient and Generalized Rayleigh Quotient
Binary Classification
Multiclass Classification
۱۱.۵.۳ Implementation
۱۱.۶ Practice: Develop and Modify Code for PCA and LDA
۱۲ Anomaly Detection
۱۲.۱ Overview
۱۲.۲ Basics of Anomaly Detection
۱۲.۳ Statistics-Based Methods
۱۲.۳.۱ ۳ Sigma
۱۲.۳.۲ Z-Score
۱۲.۳.۳ Boxplot
۱۲.۳.۴ Grubbs Hypothesis Test
۱۲.۴ Supervised Learning Methods
۱۲.۴.۱ Why Not Use Binary Classification for Anomaly Detection?
۱۲.۴.۲ Modification of Supervised Classification Methods for Anomaly Detection
۱۲.۵ Unsupervised Machine Learning Methods
۱۲.۵.۱ Overview
۱۲.۵.۲ Probabilistic Distribution Based: HBOS
۱۲.۵.۳ Distance Based: KNN
۱۲.۵.۴ Density Based: LOF, COF, INFLO, and LoOP
Local Outlier Factor (LOF)
Connectivity-Based Outlier Factor (COF)
Influenced Outlierness (INFLO)
Local Outlier Probability (LoOP)
۱۲.۵.۵ Clustering Based
۱۲.۵.۶ Tree Based
iForest
SCiForest
RRCF
Pros and Cons
۱۲.۶ Semisupervised Learning Methods
۱۲.۶.۱ Overview
۱۲.۶.۲ Autoencoder
Introduction
Preparation: Packages and Data
Model Establishment
Training and Prediction
Result Evaluation and Visualization
۱۲.۷ Anomaly Detection Issues
۱۲.۷.۱ Data Quality
۱۲.۷.۲ Imbalanced Distributions
۱۲.۷.۳ High-Dimensional Data
۱۲.۷.۴ Model Sensitivity
۱۲.۸ Practice: Implement Typical Anomaly Detection Methods
۱۳ Association Rule Learning
۱۳.۱ Overview
۱۳.۲ Basics of Association Rule Learning
۱۳.۲.۱ Definition
۱۳.۲.۲ Relationships with Other Machine Learning Topics
۱۳.۲.۳ Understanding via History
۱۳.۳ Essential Concepts of Association Rules
۱۳.۳.۱ Items, Itemsets, and Rules
۱۳.۳.۲ Support, Confidence, and Lift
۱۳.۳.۳ Association Rule Analysis Using the Concepts
۱۳.۴ Apriori
۱۳.۴.۱ Procedure
۱۳.۴.۲ Implementation with an Example
۱۳.۴.۳ Pros and Cons
۱۳.۵ FP Growth
۱۳.۵.۱ Procedure
۱۳.۵.۲ Item Header Table
۱۳.۵.۳ FP Tree
۱۳.۵.۴ Mining FP Tree for Frequent Itemsets
۱۳.۶ Eclat
۱۳.۶.۱ Procedure
۱۳.۶.۲ Implementation
۱۳.۷ Practice: Perform Association Rule Learning with Eclat
۱۴ Value-Based Reinforcement Learning
۱۴.۱ Overview
۱۴.۲ Basics of Reinforcement Learning
۱۴.۲.۱ Basic Concepts
۱۴.۲.۲ Markov Decision Process
۱۴.۲.۳ Policy Function, State Function, State-Action Function, and Reward Function
۱۴.۲.۴ Implementation of RL Environment
Implementation with OpenAI Gym
Implementation from Scratch
۱۴.۳ Bellman Equation
۱۴.۳.۱ Formulations of Bellman Equation
۱۴.۳.۲ Deduction of Bellman Equation
۱۴.۳.۳ Use of Bellman Equation in Reinforcement Learning
۱۴.۴ Value-Based RL
۱۴.۴.۱ Overview of RL Algorithms
۱۴.۴.۲ Q Learning and Sarsa
۱۴.۴.۳ Monte Carlo Method
۱۴.۵ Practice: Solve RL Problem Using Q Learning
۱۵ Policy-Based Reinforcement Learning
۱۵.۱ Overview
۱۵.۲ Policy-Based RL vs. Value-Based RL
۱۵.۳ Basic Concepts
۱۵.۴ Objective Function and Policy Gradient Theorem
۱۵.۴.۱ Objective Function
۱۵.۴.۲ Policy Gradient Theorem
۱۵.۴.۳ Simple Episodic Monte Carlo Implementation of Policy Gradient: REINFORCE V1
۱۵.۴.۴ Strategies for Improving Policy Gradient Implementation
۱۵.۵ Policy Function
۱۵.۵.۱ Linear Policy Function for Discrete Actions: Formulation 1
۱۵.۵.۲ Linear Policy Function for Discrete Actions: Formulation 2
۱۵.۵.۳ Policy Function for Continuous Actions
۱۵.۶ Common Policy Gradient Algorithms
۱۵.۶.۱ More Objective Function Formulations
۱۵.۶.۲ Simple Stepwise Monte Carlo Implementation of Policy Gradient: REINFORCE V2
۱۵.۶.۳ Actor-Critic
۱۵.۶.۴ Actor-Critic with Baseline
۱۵.۶.۵ More Policy Gradient Algorithms
۱۵.۷ Practice: Understand and Modify Policy Gradient Code for Addressing RL Problem
A Appendices
A.1 Overview
A.2 Mathematics for Machine Learning
A.2.1 Statistics
Random Variables
Probabilities
Use of Probability in Machine Learning
Probability Distributions
A.2.2 Information Theory
A.2.3 Array Operations
Matrix Operations
General Array Operations
Array Calculus
A.3 Optimization
A.3.1 Gradient-Based Methods
A.3.2 Newton's Method and Quasi-Newton's Methods
A.3.3 Conjugate Gradient Methods
A.3.4 Expectation-Maximization Methods
A.4 Evaluation Metrics
A.4.1 Overview and Basics
A.4.2 Classification: Binary
Confusion Matrix
ROC and AUC
Logarithmic Loss
A.4.3 Classification: Multiclass
Indirect Methods
Confusion Matrix
Logarithmic Loss
Kappa Coefficient
Hinge Loss
A.4.4 Classification: Multi-Label
Hamming Distance
Jaccard Similarity Coefficient
A.4.5 Regression
Root Mean Squared Error
Mean Absolute Error
Mean Squared Error
Root Mean Squared Logarithmic Error
R2 and Adjusted R2
A.4.6 Clustering
Inertia and Dunn Index
Silhouette
Davies-Bouldin Index
Calinski-Harabasz Index
Adjusted Rand Index
Adjusted Mutual Information
Bibliography
Index