Find-S algorithm, mistake-bound learning in 220—221
Find-S algorithm, PAC learning of boolean conjunctions with 212
Find-S algorithm, search of hypothesis space 27—28
Finite horizon reward 371
First-order Horn clauses 283—284 318—319 See
First-order Horn clauses in analytical learning 311
First-order Horn clauses in Prolog-EBG 313 314
First-order logic, basic definitions 285
First-order representations, applications of 275
First-order resolution rule 296—297
First-order rules 274—275 283 301 302 See
First-order rules in FOIL algorithm 285—291
First-order rules, propositional rules, comparison with 283
Fitness function 250—252 255—256 258
Fitness proportionate selection 255
Fitness sharing 259
FOCL algorithm 302
FOCL algorithm, extensions to FOIL 357
FOCL algorithm, search step alteration with prior knowledge 339—340
FOIL algorithm 286 290—291 302
FOIL algorithm, extensions in FOCL 357
FOIL algorithm, information gain measure in 289
FOIL algorithm, Learn-one-rule and sequential covering algorithms, comparison with 287
FOIL algorithm, learning first-order rules in 285—291
FOIL algorithm, post-pruning in 291
FOIL algorithm, recursive rule learning in 290
Function approximation 8
Function approximation algorithms as lookup table substitute 384
Function approximation algorithms, choice of 9—11
Functions in logic 284 285
GABIL 256—259 269
GABIL, C4.5 and AQ14 algorithms, comparison with 256 258
GABIL, extensions to 258—259
GABIL, ID5R algorithm, comparison with 258
Gain ratio 73—74
Gas See "Genetic algorithms"
Gaussian distribution See "Normal distribution"
Gaussian kernel function 238—240
General-to-specific beam search 277—279 302
General-to-specific beam search in CN2 algorithm 278
General-to-specific beam search in FOCL algorithm 357—361
General-to-specific beam search in FOIL algorithm 287 357—358
General-to-specific beam search, advantages of 281
General-to-specific ordering of hypotheses 24—25 45—46 See
Generalization accuracy in neural networks 110—111
Generalizer 12 13
Generate-and-test beam search 250
Generate-and-test beam search, example-driven search, comparison with 281
Generate-and-test beam search, inverse entailment operators, comparison with 299
Generate-and-test beam search, inverse resolution, comparison with 298—299
Genetic algorithms 249—270
Genetic algorithms, advantages of 250
Genetic algorithms, applications of 256 269
Genetic algorithms, fitness function in 255—256
Genetic algorithms, limitations of 259
Genetic algorithms, parallelization of 268
Genetic algorithms, representation of hypotheses 252—253
Genetic algorithms, search of hypothesis space 259 268—269
Genetic operators 252—255 257 261—262
Genetic programming 250 262—266 269
Genetic programming, applications of 265 269
Genetic programming, performance of 266
Genetic programming, representation in 262—263
Gibbs algorithm 176
Global method 234
GOLEM 281
GP See "Genetic programming"
Gradient ascent search 170—171
Gradient ascent search in Bayesian belief networks 188—190
Gradient descent search 89—91 93 97 115—116 123
Gradient descent search in EBNN algorithm 339
Gradient descent search, least-squared error hypothesis in 167
Gradient descent search, limitations of 92
Gradient descent search, weight update rule 91—92 237
Gradient descent search, weight update rule, stochastic approximation to 92—94 98—100 104—105 107—108
Gradient of error 91
Greedy search in Prolog-EBG 323
Greedy search in sequential covering algorithms 276—278
GRENDEL program 303
Ground literal 285
Halving algorithm 223
Halving algorithm, mistake-bound learning in 221—222
handwriting recognition 3—4
Handwriting recognition, Backpropagation algorithm in 81
Handwriting recognition, TangentProp algorithm in 348—349
Head of Horn clause 285
Hidden layer representations, discovery by Backpropagation algorithm 106—109 123
Hidden units in face recognition task 115—117
Hidden units, Backpropagation weight tuning rule for 103
Hidden units, Cascade-Correlation algorithm, addition by 121—123
Hidden units, choice in radial basis function networks 239—240
Hill-climbing search in FOIL algorithm 286 287
Hill-climbing search in genetic algorithms 268
Hill-climbing search in ID3 algorithm 60—61
Hoeffding bounds 210—211
Horn clauses 284 285
Horn clauses, first-order See "First-order Horn clauses"
Human learning, explanations in 309
Human learning, prior knowledge in 330
Hypotheses See also "Discrete-valued hypotheses" "General-to-specific "Hypothesis
Hypotheses, error differences between two 143—144
Hypotheses, estimation of accuracy 129—130
Hypotheses, estimation of accuracy, bias and variance in estimate 129 151 152
Hypotheses, estimation of accuracy, errors in 129—131 151
Hypotheses, evaluation of 128—129
Hypotheses, justification of, in inductive vs. analytical learning 334—336
Hypotheses, representations of 23
Hypotheses, testing of 144—145
Hypothesis space 14—15
Hypothesis space search by Backpropagation algorithm 97 106 122—123
Hypothesis space search by Backpropagation algorithm, comparison with decision tree learning 106
Hypothesis space search by Backpropagation algorithm, comparison with KBANN and TangentProp algorithms 350—351
Hypothesis space search by Find-S algorithm 27—28
Hypothesis space search by FOIL algorithm 286—287 357—361
Hypothesis space search by genetic algorithms 250 259
Hypothesis space search by gradient descent 90—91
Hypothesis space search by ID3 algorithm 60—62 64 76
Hypothesis space search by KBANN algorithm 346
Hypothesis space search by Learn-one-rule 277
Hypothesis space search by learning algorithms 24
Hypothesis space search in concept learning 23—25 46—47
Hypothesis space search in machine learning 14—15 18
Hypothesis space search, by Candidate-Elimination algorithm 64
Hypothesis space search, constraints on 302—303
Hypothesis space search, use of prior knowledge 339—340 362
Hypothesis space, bias in 40—42 46 129
Hypothesis space, finite, sample complexity for 207—214 225
Hypothesis space, infinite, sample complexity for 214—220
Hypothesis space, VC dimension of 214—217
ID3 algorithm 55—64 77
ID3 algorithm, backtracking and 62
ID3 algorithm, Candidate-Elimination algorithm, comparison with 61—62
ID3 algorithm, choice of attributes in 280—281
ID3 algorithm, choice of decision tree 63
ID3 algorithm, cost-sensitive measures 75—76
ID3 algorithm, extensions to 77 See
ID3 algorithm, inductive bias of 63—64 76
ID3 algorithm, Learn-one-rule, search comparison with 277
ID3 algorithm, limitations of 61—62
ID3 algorithm, overfitting in 67—68
ID3 algorithm, search of hypothesis space 60—62 64 76
ID3 algorithm, sequential covering algorithms, comparison with 280—281
ID3 algorithm, specialized for concept learning 56
ID3 algorithm, use of information gain in 58—60
ID5R algorithm, comparison with GABIL 258
ILP See "Inductive logic programming"
Image encoding in face recognition 114
Imperfect domain theory in EBNN algorithm 356
Imperfect domain theory in explanation-based learning 330
Imperfect domain theory in FOCL algorithm 360
| Imperfect domain theory in KBANN algorithm 344—345
Incremental explanation methods 328
Incremental gradient descent See "Stochastic gradient descent"
Incremental Version Space Merging algorithm 47
Inductive bias 39—45 137—138 See "Preference "Restriction
Inductive bias in decision tree learning 63—66
Inductive bias in explanation-based learning 322—323
Inductive bias of Backpropagation algorithm 106
Inductive bias of Candidate-Elimination algorithm 43—46 63—64
Inductive bias of Find-S algorithm 45
Inductive bias of ID3 algorithm 63—64 76
Inductive bias of inductive learning algorithms 42—46
Inductive bias of k-Nearest Neighbor algorithm 234
Inductive bias of LMS algorithm 64
Inductive bias of Rote-Learner algorithm 44—45
Inductive bias, bias-free learning 40—42
Inductive bias, definition of 43
Inductive inference See "Inductive learning"
Inductive learning 42 307—308 See "Genetic "Inductive "Neural
Inductive learning hypothesis 23
Inductive learning, analytical learning, comparison with 310 328—329 334—336 362
Inductive learning, inductive bias in 42—46
Inductive logic programming 275 291
Inductive logic programming, Prolog-EBG, comparison with 322
Inductive-analytical learning 334—363
Inductive-analytical learning, advantages of 362
Inductive-analytical learning, explanation-based learning and 330
Inductive-analytical learning, learning problem 337—338
Inductive-analytical learning, prior knowledge methods to alter search 339—340 362
Inductive-analytical learning, properties of ideal systems 337
Inductive-analytical learning, weighting of components in EBNN algorithm 351—352 355
Inductive-analytical learning, weighting prior knowledge in 338
Information gain 73
Information gain in FOIL algorithm 289
Information gain in ID3 algorithm 55 58—60
Information gain, definition of 57—58
Information theory, influence on machine learning 4
Information theory, Minimum Description Length principle and 172
Initialize-the-hypothesis approach 339—346
Initialize-the-hypothesis approach, Bayesian belief networks in 346
Instance-based learning 230—247 See "k-Nearest "Locally
Instance-based learning, advantages 245—246
Instance-based learning, case-based reasoning, comparison with other methods 240
Instance-based learning, limitations of 231
Inverse entailment 292 302
Inverse entailment in Progol 300—302
Inverse entailment, first-order 297
Inverse entailment, generate-and-test beam search, comparison with 299
Inverse resolution 294—296 302
Inverse resolution, first-order 297—298
Inverse resolution, generate-and-test beam search, comparison with 298—299
Inverse resolution, limitations of 300
Inverted deduction 291—293
Jacobian 354
Job-shop scheduling, genetic algorithms in 256
Joint probability distribution in Bayesian belief networks 185—187
k-fold cross-validation 112 147 150
k-means problem 191—193
k-means problem, derivation of EM algorithm for 195—196
k-nearest neighbor algorithm 231—233 246
k-Nearest Neighbor algorithm, applications of 234
k-Nearest Neighbor algorithm, cross-validation in 235
k-Nearest Neighbor algorithm, decision tree and rule learning, comparison with 235
k-Nearest Neighbor algorithm, distance-weighted 233—234
k-Nearest Neighbor algorithm, inductive bias of 234
k-Nearest Neighbor algorithm, memory indexing in 236
k-term CNF expressions 213—214
k-term DNF expressions 213—214
K2 algorithm 190—191
KBANN algorithm 340—347 362 387
KBANN algorithm, advantages of 344
KBANN algorithm, Backpropagation algorithm, comparison with 344—345
KBANN algorithm, Backpropagation weight update rule in 343—344
KBANN algorithm, hypothesis space search by Backpropagation and TangentProp, comparison with 350—351
KBANN algorithm, limitations of 345
KBANN algorithm, prior knowledge in 339
kd-tree 236
Kernel function 236 238 246
Kernel function, Gaussian See "Gaussian kernel function"
Knowledge compilation 320
Knowledge level learning 323—325
Knowledge reformulation 320
Knowledge-Based Artificial Neural Network (KBANN) algorithm See "KBANN algorithm"
Lamarckian evolution 266
Language bias See "Restriction bias"
Lazy explanation methods 328
Lazy learning methods, comparison with eager learning 244—245
Learn-one-rule algorithm, FOIL algorithm, comparison with 287
Learn-one-rule algorithm, ID3 algorithm, search comparison with 277
Learn-one-rule algorithm, rule performance in 282
Learn-one-rule algorithm, rule post-pruning in 281
Learn-one-rule algorithm, variations of 279—280 286
Learning algorithms, consistent learners 162—163
Learning algorithms, design of 9—11 17
Learning algorithms, domain-independent 336
Learning algorithms, error differences between two 145—151
Learning algorithms, search of hypothesis space 24
Learning problems 2—5 17
Learning problems in inductive-analytical learning 337—338
Learning problems, computational theory of 201—202
Learning rate 88 91
Learning systems, design of 5—14 17
Learning systems, program modules 11—14
Learning, human See "Human learning"
Learning, machine See "Machine learning"
Learn_naive_Bayes_text 182—183
Least mean squares algorithm See "LMS algorithm"
Least-squared error hypothesis, classifiers for 198
Least-squared error hypothesis, gradient descent in 167
Least-squared error hypothesis, maximum likelihood (ML) hypothesis and 164—167
Leave-one-out cross-validation 235
Legal case reasoning, case-based reasoning in 240
Lemma-Enumerator algorithm 324
Lifelong learning 370
Line search 119
Linear programming, as weight update algorithm 95
Linearly separable sets 86 89 95
List-Then-Eliminate algorithm 30
Literal 284 285
LMS algorithm 11 15
LMS algorithm, inductive bias of 64
LMS weight update rule See "Delta rule"
Local method 234
Locally weighted regression 231 236—238 246
Locally weighted regression, limitations of 238
Locally weighted regression, weight update rules in 237—238
logical constants 284 285
Logical terms 284 285
Logistic function 96 104
Lookup table, function approximation algorithms as substitute 384
Lookup table, neural network as substitute 384
Lower bound on sample complexity 217—218
m-estimate of probability 179—180 198 282
Machine learning 15 See
Machine learning, applications 5 17
Machine learning, definition of 2
Machine learning, influence of other disciplines on 4 17
Machine learning, search of hypothesis space 14—15 18
Manufacturing process control 17
MAP hypothesis See "Maximum a posteriori hypothesis"
MAP Learning algorithm, Brute-Force See "Brute-Force MAP Learning algorithm"
Markov decision processes (MDP) 370 387
Markov decision processes (MDP), applications of 386
Markus 302
Marvin 302
Maximally general hypotheses, computation by Candidate-Elimination algorithm 31 46
Maximally specific hypotheses, computation by Candidate-Elimination algorithm 31 46
Maximally specific hypotheses, computation by Find-S algorithm 26—28 62—63
Maximum a posteriori (MAP) hypothesis 157 197 See
Maximum a posteriori (MAP) hypothesis, naive Bayes classifier and 178
|
|
|
Î ïðîåêòå