On the optimal weight of a perceptron with Gaussian data and arbitrary nonlinearity
MetadataShow full item record
In this correspondence we investigate the solution to the following problem: Find the optimal weighted sum of given signals when the optimality criteria is the expected value of a function of this sum and a given "training" signal. The optimality criteria can be a nonlinear function from a very large family of possible functions. A number of interesting cases fall under this general framework, such as a single layer perceptron with any of the commonly used nonlinearities, the LMS, the LMF or higher moments, or the various sign algorithms. Assuming the signals to be jointly Gaussian we show that the optimal solution, when it exits, is always collinear with the well-known Wiener solution, and only its scaling factor depends on the particular functions chosen. We also present necessary constructive conditions for the existance of the optimal solution.
Showing items related by title, author, creator and subject.
Gilchrist, Richard B. (Cambridge, Massachusetts: Massachusetts Institute of Technology, 1964-01);Dynamic programming is employed to obtain a solution to the problem of controlling a nonlinear system in an optimal fashion, subject to a quadratic performance index. The technique sued is similar to that given by Merriam ...
Guthrie, Katherine H. (Monterey, California: Naval Postgraduate School, 2017-06);A cascade heuristic appeals when we are faced with a monolithic optimization model exhibiting more decision variables and/or constraints than can be accommodated by computers and/or optimization software available. This ...
Lazzaro, Gary L. (Monterey, California: Naval Postgraduate School, 2016-06);The optimal defense and operation of networks against worst-case attack is an important problem for military analysts. We review development of existing solutions for the Defender-Attacker-Defender (DAD) tri-level optimization ...