bp算法

㈠ BP算法的介绍

BP算法，误差反向传播（Error Back Propagation, BP）算法。BP算法的基本思想是，学习过程由信号的正向传播与误差的反向传播两个过程组成。由于多层前馈网络的训练经常采用误差反向传播算法，人们也常把将多层前馈网络直接称为BP网络。

㈡ 关于BP算法，真正明白BP算法的进！

一、隐藏层顾名思义，他的输出者就是给别的层使用的，一般隐藏着不给人看。
二、干什么用的……这个都可以从AI发展史来说了，就是早期的神经网络是单层，于是有人证明这种的学习能力有限，有些问题，比如异或就不能解决，后来，就有人提出了多层神经网络，增强了神经网络的学习能力。其中非输出层都是隐藏层（好像是这样），最根本的作用就是增加神经网络的学习能力。最直接的作用就是把接收到的输入信号产生一个输出给下一层。

㈢ 菜鸟matlab神经网络BP算法问题

你发完这个，有什么问题吗？

㈣ 谁能给我讲讲BP算法中的输入和输出是什么概念

第1章 BP神经网络的数据分类——语音特征信号分类第2章 BP神经网络的非线性系统建模——非线性函数拟合《MATLAB 神经网络43个案例分析》目录http://www.matlabsky.com/thread-37140-1-1.html

㈤ BP算法、BP神经网络、遗传算法、神经网络这四者之间的关系

这四个都属于人工智能算法的范畴。其中BP算法、BP神经网络和神经网络
属于神经网络这个大类。遗传算法为进化算法这个大类。
神经网络模拟人类大脑神经计算过程，可以实现高度非线性的预测和计算，主要用于非线性拟合，识别，特点是需要“训练”，给一些输入，告诉他正确的输出。若干次后，再给新的输入，神经网络就能正确的预测对于的输出。神经网络广泛的运用在模式识别，故障诊断中。BP算法和BP神经网络是神经网络的改进版，修正了一些神经网络的缺点。
遗传算法属于进化算法，模拟大自然生物进化的过程：优胜略汰。个体不断进化，只有高质量的个体（目标函数最小（大））才能进入下一代的繁殖。如此往复，最终找到全局最优值。遗传算法能够很好的解决常规优化算法无法解决的高度非线性优化问题，广泛应用在各行各业中。差分进化，蚁群算法，粒子群算法等都属于进化算法，只是模拟的生物群体对象不一样而已。

㈥ BP算法的实现步骤

BP算法实现步骤（软件）：
1）初始化
2）输入训练样本对，计算各层输出
3）计算网络输出误差
4）计算各层误差信号
5）调整各层权值
6）检查网络总误差是否达到精度要求
满足，则训练结束；不满足，则返回步骤2）
3、多层感知器（基于BP算法）的主要能力：
1）非线性映射：足够多样本－>学习训练
能学习和存储大量输入－输出模式映射关系。只要能提供足够多的样本模式对供BP网络进行学习训练，它便能完成由n维输入空间到m维输出空间的非线性映射。
2）泛化：输入新样本（训练时未有）－>完成正确的输入、输出映射
3）容错：个别样本误差不能左右对权矩阵的调整
4、标准BP算法的缺陷：
1）易形成局部极小（属贪婪算法，局部最优)而得不到全局最优；
2）训练次数多使得学习效率低下，收敛速度慢（需做大量运算）；
3）隐节点的选取缺乏理论支持；
4）训练时学习新样本有遗忘旧样本趋势。
注3：改进算法—增加动量项、自适应调整学习速率（这个似乎不错)及引入陡度因子

㈦ 如何理解BP学习算法 追加悬赏

又称为BP网络.BP学习算法是一种有效的学习方法,但由于在权值调整上采用梯度下降法作为优化算法,容易陷入局部最小,不能保证得到全局最优解。

㈧ 监督学习是不是bp算法

监督学习是你给定的数据它们都有标签，然后训练完了之后你再用别的不带标签的数据输进去，系统给你算出一个标签出来，这里的标签可以是离散的，也可以是连续的
BP算法是优化神经网络的一种算法，它是利用链式法则和反向求导来实现的

两个性质不一样

㈨ BP算法及其改进

传统的BP算法及其改进算法的一个很大缺点是：由于其误差目标函数对于待学习的连接权值来说非凸的，存在局部最小点，对网络进行训练时，这些算法的权值一旦落入权值空间的局部最小点就很难跳出，因而无法达到全局最小点(即最优点)而使得网络训练失败。针对这些缺陷，根据凸函数及其共轭的性质，利用Fenchel不等式，使用约束优化理论中的罚函数方法构造出了带有惩罚项的新误差目标函数。

用新的目标函数对前馈神经网络进行优化训练时，隐层输出也作为被优化变量。这个目标函数的主要特点有：
1.固定隐层输出，该目标函数对连接权值来说是凸的；固定连接权值，对隐层输出来说是凸的。这样在对连接权值和隐层输出进行交替优化时，它们所面对的目标函数都是凸函数，不存在局部最小的问题，算法对于初始权值的敏感性降低；
2.由于惩罚因子是逐渐增大的，使得权值的搜索空间变得比较大，从而对于大规模的网络也能够训练，在一定程度上降低了训练过程陷入局部最小的可能性。

这些特性能够在很大程度上有效地克服以往前馈网络的训练算法易于陷入局部最小而使网络训练失败的重大缺陷，也为利用凸优化理论研究前馈神经网络的学习算法开创了一个新思路。在网络训练时，可以对连接权值和隐层输出进行交替优化。把这种新算法应用到前馈神经网络训练学习中，在学习速度、泛化能力、网络训练成功率等多方面均优于传统训练算法，如经典的BP算法。数值试验也表明了这一新算法的有效性。

本文通过典型的BP算法与新算法的比较，得到了二者之间相互关系的初步结论。从理论上证明了当惩罚因子趋于正无穷大时新算法就是BP算法，并且用数值试验说明了惩罚因子在网络训练算法中的作用和意义。对于三层前馈神经网络来说，惩罚因子较小时，隐层神经元局部梯度的可变范围大，有利于连接权值的更新；惩罚因子较大时，隐层神经元局部梯度的可变范围小，不利于连接权值的更新，但能提高网络训练精度。这说明了在网络训练过程中惩罚因子为何从小到大变化的原因,也说明了新算法的可行性而BP算法则时有无法更新连接权值的重大缺陷。

矿体预测在矿床地质中占有重要地位，由于输入样本量大，用以往前馈网络算法进行矿体预测效果不佳。本文把前馈网络新算法应用到矿体预测中，取得了良好的预期效果。

本文最后指出了新算法的优点，并指出了有待改进的地方。

关键词：前馈神经网络，凸优化理论，训练算法，矿体预测，应用

Feed forward Neural Networks Training Algorithm Based on Convex Optimization and Its Application in Deposit Forcasting
JIA Wen-chen (Computer Application)
Directed by YE Shi-wei

Abstract

The paper studies primarily the application of convex optimization theory and algorithm for feed forward neural networks’ training and convergence performance.

It reviews the history of feed forward neural networks, points out that the training of feed forward neural networks is essentially a non-linear problem and introces BP algorithm, its advantages as well as disadvantages and previous improvements for it. One of the big disadvantages of BP algorithm and its improvement algorithms is: because its error target function is non-convex in the weight values between neurons in different layers and exists local minimum point, thus, if the weight values enter local minimum point in weight values space when network is trained, it is difficult to skip local minimum point and reach the global minimum point (i.e. the most optimal point).If this happening, the training of networks will be unsuccessful. To overcome these essential disadvantages, the paper constructs a new error target function including restriction item according to convex function, Fenchel inequality in the conjugate of convex function and punishment function method in restriction optimization theory.
When feed forward neural networks based on the new target function is being trained, hidden layers’ outputs are seen as optimization variables. The main characteristics of the new target function are as follows:

1.With fixed hidden layers’ outputs, the new target function is convex in connecting weight variables; with fixed connecting weight values, the new target function is convex in hidden layers’ outputs. Thus, when connecting weight values and hidden layers’ outputs are optimized alternately, the new target function is convex in them, doesn’t exist local minimum point, and the algorithm’s sensitiveness is reced for original weight values .
2.Because the punishment factor is increased graally, weight values ’ searching space gets much bigger, so big networks can be trained and the possibility of entering local minimum point can be reced to a certain extent in network training process.

Using these characteristics can overcome efficiently in the former feed forward neural networks’ training algorithms the big disadvantage that networks training enters local minimum point easily. This creats a new idea for feed forward neural networks’ learning algorithms by using convex optimization theory .In networks training, connecting weight variables and hidden layer outputs can be optimized alternately. The new algorithm is much better than traditional algorithms for feed forward neural networks. The numerical experiments show that the new algorithm is successful.

By comparing the new algorithm with the traditional ones, a primary conclusion of their relationship is reached. It is proved theoretically that when the punishment factor nears infinity, the new algorithm is BP algorithm yet. The meaning and function of the punishment factor are also explained by numerical experiments. For three-layer feed forward neural networks, when the punishment factor is smaller, hidden layer outputs’ variable range is bigger and this is in favor to updating of the connecting weights values, when the punishment factor is bigger, hidden layer outputs’ variable range is smaller and this is not in favor to updating of the connecting weights values but it can improve precision of networks. This explains the reason that the punishment factor should be increased graally in networks training process. It also explains feasibility of the new algorithm and BP algorithm’s disadvantage that connecting weigh values can not be updated sometimes.

Deposit forecasting is very important in deposit geology. The previous algorithms’ effect is not good in deposit forecasting because of much more input samples. The paper applies the new algorithm to deposit forecasting and expectant result is reached.
The paper points out the new algorithm’s strongpoint as well as to-be-improved places in the end.

Keywords: feed forward neural networks, convex optimization theory, training algorithm, deposit forecasting, application

传统的BP算法及其改进算法的一个很大缺点是：由于其误差目标函数对于待学习的连接权值来说非凸的，存在局部最小点，对网络进行训练时，这些算法的权值一旦落入权值空间的局部最小点就很难跳出，因而无法达到全局最小点(即最优点)而使得网络训练失败。针对这些缺陷，根据凸函数及其共轭的性质，利用Fenchel不等式，使用约束优化理论中的罚函数方法构造出了带有惩罚项的新误差目标函数。

用新的目标函数对前馈神经网络进行优化训练时，隐层输出也作为被优化变量。这个目标函数的主要特点有：
1.固定隐层输出，该目标函数对连接权值来说是凸的；固定连接权值，对隐层输出来说是凸的。这样在对连接权值和隐层输出进行交替优化时，它们所面对的目标函数都是凸函数，不存在局部最小的问题，算法对于初始权值的敏感性降低；
2.由于惩罚因子是逐渐增大的，使得权值的搜索空间变得比较大，从而对于大规模的网络也能够训练，在一定程度上降低了训练过程陷入局部最小的可能性。

这些特性能够在很大程度上有效地克服以往前馈网络的训练算法易于陷入局部最小而使网络训练失败的重大缺陷，也为利用凸优化理论研究前馈神经网络的学习算法开创了一个新思路。在网络训练时，可以对连接权值和隐层输出进行交替优化。把这种新算法应用到前馈神经网络训练学习中，在学习速度、泛化能力、网络训练成功率等多方面均优于传统训练算法，如经典的BP算法。数值试验也表明了这一新算法的有效性。

本文通过典型的BP算法与新算法的比较，得到了二者之间相互关系的初步结论。从理论上证明了当惩罚因子趋于正无穷大时新算法就是BP算法，并且用数值试验说明了惩罚因子在网络训练算法中的作用和意义。对于三层前馈神经网络来说，惩罚因子较小时，隐层神经元局部梯度的可变范围大，有利于连接权值的更新；惩罚因子较大时，隐层神经元局部梯度的可变范围小，不利于连接权值的更新，但能提高网络训练精度。这说明了在网络训练过程中惩罚因子为何从小到大变化的原因,也说明了新算法的可行性而BP算法则时有无法更新连接权值的重大缺陷。

矿体预测在矿床地质中占有重要地位，由于输入样本量大，用以往前馈网络算法进行矿体预测效果不佳。本文把前馈网络新算法应用到矿体预测中，取得了良好的预期效果。

本文最后指出了新算法的优点，并指出了有待改进的地方。

关键词：前馈神经网络，凸优化理论，训练算法，矿体预测，应用

Feed forward Neural Networks Training Algorithm Based on Convex Optimization and Its Application in Deposit Forcasting
JIA Wen-chen (Computer Application)
Directed by YE Shi-wei

Abstract

The paper studies primarily the application of convex optimization theory and algorithm for feed forward neural networks’ training and convergence performance.

It reviews the history of feed forward neural networks, points out that the training of feed forward neural networks is essentially a non-linear problem and introces BP algorithm, its advantages as well as disadvantages and previous improvements for it. One of the big disadvantages of BP algorithm and its improvement algorithms is: because its error target function is non-convex in the weight values between neurons in different layers and exists local minimum point, thus, if the weight values enter local minimum point in weight values space when network is trained, it is difficult to skip local minimum point and reach the global minimum point (i.e. the most optimal point).If this happening, the training of networks will be unsuccessful. To overcome these essential disadvantages, the paper constructs a new error target function including restriction item according to convex function, Fenchel inequality in the conjugate of convex function and punishment function method in restriction optimization theory.
When feed forward neural networks based on the new target function is being trained, hidden layers’ outputs are seen as optimization variables. The main characteristics of the new target function are as follows:

1.With fixed hidden layers’ outputs, the new target function is convex in connecting weight variables; with fixed connecting weight values, the new target function is convex in hidden layers’ outputs. Thus, when connecting weight values and hidden layers’ outputs are optimized alternately, the new target function is convex in them, doesn’t exist local minimum point, and the algorithm’s sensitiveness is reced for original weight values .
2.Because the punishment factor is increased graally, weight values ’ searching space gets much bigger, so big networks can be trained and the possibility of entering local minimum point can be reced to a certain extent in network training process.

Using these characteristics can overcome efficiently in the former feed forward neural networks’ training algorithms the big disadvantage that networks training enters local minimum point easily. This creats a new idea for feed forward neural networks’ learning algorithms by using convex optimization theory .In networks training, connecting weight variables and hidden layer outputs can be optimized alternately. The new algorithm is much better than traditional algorithms for feed forward neural networks. The numerical experiments show that the new algorithm is successful.

By comparing the new algorithm with the traditional ones, a primary conclusion of their relationship is reached. It is proved theoretically that when the punishment factor nears infinity, the new algorithm is BP algorithm yet. The meaning and function of the punishment factor are also explained by numerical experiments. For three-layer feed forward neural networks, when the punishment factor is smaller, hidden layer outputs’ variable range is bigger and this is in favor to updating of the connecting weights values, when the punishment factor is bigger, hidden layer outputs’ variable range is smaller and this is not in favor to updating of the connecting weights values but it can improve precision of networks. This explains the reason that the punishment factor should be increased graally in networks training process. It also explains feasibility of the new algorithm and BP algorithm’s disadvantage that connecting weigh values can not be updated sometimes.

Deposit forecasting is very important in deposit geology. The previous algorithms’ effect is not good in deposit forecasting because of much more input samples. The paper applies the new algorithm to deposit forecasting and expectant result is reached.
The paper points out the new algorithm’s strongpoint as well as to-be-improved places in the end.

Keywords: feed forward neural networks, convex optimization theory, training algorithm, deposit forecasting, application

BP算法及其改进

2.1 BP算法步骤

1°随机抽取初始权值ω0；

2°输入学习样本对（Xp,Yp），学习速率η，误差水平ε；

3°依次计算各层结点输出opi,opj,opk；

4°修正权值ωk+1=ωk+ηpk,其中pk=,ωk为第k次迭代权变量；

5°若误差E<ε停止，否则转3°。

2.2 最优步长ηk的确定

在上面的算法中，学习速率η实质上是一个沿负梯度方向的步长因子，在每一次迭代中如何确定一个最优步长ηk，使其误差值下降最快，则是典型的一维搜索问题，即E(ωk+ηkpk)=(ωk+ηpk)。令Φ(η)=E(ωk+ηpk)，则Φ′(η)=dE(ωk+ηpk)/dη=E(ωk+ηpk)Tpk。若ηk为(η)的极小值点，则Φ′(ηk)=0，即E(ωk+ηpk)Tpk=-pTk+1pk=0。确定ηk的算法步骤如下

1°给定η0=0,h=0.01,ε0=0.00001；

2°计算Φ′(η0),若Φ′(η0)=0,则令ηk=η0，停止计算；

3°令h=2h, η1=η0+h；

4°计算Φ′(η1)，若Φ′(η1)=0，则令ηk=η1，停止计算；

若Φ′(η1)>0，则令a=η0,b=η1；若Φ′(η1)<0，则令η0=η1，转3°；

5°计算Φ′(a)，若Φ′(a)=0，则ηk=a，停止计算；

6°计算Φ′(b)，若Φ′(b)=0，则ηk=b，停止计算；

7°计算Φ′(a+b/2),若Φ′(a+b/2)=0，则ηk=a+b/2，停止计算；

若Φ′(a+b/2)<0,则令a=a+b/2;若Φ′(a+b/2)>0，则令b=a+b/2

8°若｜a-b｜<ε0，则令，ηk=a+b/2,停止计算，否则转7°。

2.3 改进BP算法的特点分析

在上述改进的BP算法中，对学习速率η的选取不再由用户自己确定，而是在每次迭代过程中让计算机自动寻找最优步长ηk。而确定ηk的算法中，首先给定η0=0，由定义Φ(η)=E(ωk+ηpk)知，Φ′(η)=dE(ωk+ηpk)/dη=E(ωk+ηpk)Tpk，即Φ′(η0)=-pTkpk≤0。若Φ′(η0)=0，则表明此时下降方向pk为零向量，也即已达到局部极值点，否则必有Φ′(η0)<0，而对于一维函数Φ(η)的性质可知，Φ′(η0)<0则在η0=0的局部范围内函数为减函数。故在每一次迭代过程中给η0赋初值0是合理的。

改进后的BP算法与原BP算法相比有两处变化，即步骤2°中不需给定学习速率η的值；另外在每一次修正权值之前，即步骤4°前已计算出最优步长ηk。

㈩ BP算法的简介

1）正向传播：输入样本－>输入层－>各隐层（处理）－>输出层
注1：若输出层实际输出与期望输出（教师信号）不符，则转入2）（误差反向传播过程）
2）误差反向传播：输出误差（某种形式）－>隐层（逐层）－>输入层
其主要目的是通过将输出误差反传，将误差分摊给各层所有单元，从而获得各层单元的误差信号，进而修正各单元的权值（其过程，是一个权值调整的过程）。
注2：权值调整的过程，也就是网络的学习训练过程（学习也就是这么的由来，权值调整）。