| [9] 标题： |
| 发表评论人：[游客]coolboy [2009-12-31 9:46:22] ip:173.67.2.* |
The above discussion has been summarized in the following link:
http://bbs.lasg.ac.cn/bbs/thread-45026-1-1.html
followed by additional discussion on the ergodic theorem.
博主回复：Thjank you very much for letting me know the exsitence of these other blog pages. I live a very ignorant life as far as blogs and bloggers go (in fact science net is the only blog pages I read).
|
|
| [8] 标题： |
| 发表评论人：[游客]coolboy [2009-12-28 12:57:08] ip:173.67.2.* |
Well, well, I believe I learnt at least one optimization technique for discrete or categorical variables when I was still in high school many years ago. Of course, this was not because I was somehow special at that time. There were many, many Chinese good high school students, general technicians or even ordinary workers who learnt the same technique around the same time. This was all because a great and well-known Chinese mathematician Hua, Luogeng (华罗庚) who spent great efforts on popularizing the optimization methods (优选法) in China in the special period of the so-called Great Culture Revolution. I have only a few Chinese books on my bookshelves but one of those happens to be a book I read very seriously right after its publication and when I was truly a cool boy:
《正交试验法》编写组，1976：正交试验法。国防工业出版社，210页，定价：0.65元。
The method of orthogonal designs was also briefly introduced in Chapter 17 in the above mentioned Spall’s (2003) book. One good thing about Spall’s book is that it “focuses on methods that have a solid theoretical foundation and that have a track record of effectiveness in a broad range of practical applications.” A more recent book on orthogonal designs is:
方开泰，马长兴，2001：正交与均匀试验设计。科学出版社，248页，定价：18.00元，
in which the authors indicated that there exists an equivalence between orthogonal designs and D-Optimality.
I came to USA as a graduate student many years ago and immediately met some Chinese visiting scholars (majoring in geology and fishery) who presented their researches to their American colleagues. They were very surprised that none of the Americans knew the 优选法 (optimization methods) that were so popular in China and almost everyone in China knew it. I recalled that at the old time though people always associated 华罗庚 with 优选法 but people only said 华罗庚推广优选法 and hardly anyone had said 华罗庚发明优选法. Later, I discovered that the field had a different name in English for this special field of the optimization methods (优选法): Experimental Designs.
博主回复：Yes, "experimental design" is ONE of the many optimization method for discrete optimization. There are many examples of independent and redundant discoveries in science throughout history. To receive proper and deserved credit involve complex social conditions prevailing.
|
|
| [7] 标题： |
| 发表评论人：[游客]coolboy [2009-12-24 13:30:06] ip:173.67.2.* |
In terms of the relative importance of determinant versus randomness, we probably can divide the stochastic optimization problems into two categories: (1) those where the determinate processes are dominant with small modifications coming from randomness and (2) those we know little about precise laws governing their changes due to randomness that makes the major contributions to system variations. One well-known example in the first category is the change and control of the state of a spacecraft (or any flying object) where stochastic uncertainties only slightly modify the known dynamical system. In this category, the Kalman filter is always the best method/approach to solve the stochastic optimization problems BECAUSE the Kalman filter fully utilizes the available knowledge/resource of the determinate dynamical system. On the other hand, if one likes to optimize a network of traffic flows by appropriately setting the timings of the traffic lights at all the interactions there is hardly any determinate law that will tell us, as a first order approximation, how the flow will evolve with time. In this case, the gradient needs to be evaluated entirely by the stochastic approximations.
Wish you and your family Merry Christmas and a happy holiday season!
博主回复：I do not wish to start a complete discussion of stochastic optimization here. But it should be pointed out that so far your comments apply only to problems involving continuous vairables where gradient and other local improvement ideas apply. There is a whole new category of stochastic optimization problems invovling discrete or categorical variables where general search and very different methodology must be used.
You have a happy holidays also. Best.
|
|
| [6] 标题： |
| 发表评论人：[游客]coolboy [2009-12-24 9:21:21] ip:173.67.2.* |
| Correction: It should be "Chapter 7 of the above book contains..." rather than "Chapter 6 of the above book contains...".
|
|
| [5] 标题： |
| 发表评论人：[游客]coolboy [2009-12-23 13:13:38] ip:173.67.2.* |
So far, all the discussions have been limited to a qualitative level though both professor Ho and I have used some scientific terminologies. Since many readers of this blog are scientists, it is also worthwhile pointing to the direction of how to implement the above ideas in a precise or quantitative way. I put forward a similar argument in a recent discussion on a hot topic of global warming:
关于全球气候变暖
http://bbs.lasg.ac.cn/bbs/thread-42150-3-1.html
Coolboy:【So far, we have being talking about hydrological cycle and precipitation extremes in a quantitative or a scientific way in which numbers or logical reasoning or facts are the most important. In the early stage of the discussion, we have also mentioned the critical difference between scientific research and philosophical arguments. Therefore, it might be interesting or solely for the purpose of comparison to see a few pieces of writings on hydrological cycle and precipitation extremes in a qualitative manner or from a philosophical point of view. 】
I think one very useful reference that gives great details on modern “stochastic optimization” is the following book:
Spall, J. C., 2003: Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control. John Wiley & Sons, Inc., New York, 595 pp.
Chapter 6 of the above book contains the algorithm of implementing the original idea that “evaluates the early stage gradient with little accuracy but great efficiency, and gradually increases its accuracy as the iteration solution approaches B.
博主回复：You previous three comments basically re-iterates the essence of the method of "stochastic approximation" which is present in almost all stochastic continuous variable optimization methoids including the Kalman filter. I have no disagreement here.
|
|
| [4] 标题： |
| 发表评论人：[游客]coolboy [2009-12-23 13:12:25] ip:173.67.2.* |
The next question is: what is or do we have a recipe/algorithm for optimal searching? The answer to this question has also been given in another comment of mine on this blog:
Warren Buffett’s Ten Rules for Success
http://www.sciencenet.cn/m/user_content.aspx?id=38150
[4]Coolboy:【When we were in college, we learnt a lesson in our English class entitled “How to Study” that introduced an SQ3R strategy on how to study efficiently: Survey, Question, Read, Recite, and Revise. This SQ3R strategy can also be applied to many other things including how to find a good girlfriend for a boy or how to find a good boyfriend for a girl. For example, you first go through a “survey” process by excluding those who are too old or too young, too this or too that. Then you “question”, casually asking “are you from North or South?” “What color do you like most?” “Do you like seafood or can you cook a nice fish dish?” etc. The measure of how much you can “read” her mind or vice versa should give you an idea of how much commonality you two have. You then review how much you are able to “recite” what you have told her in previous conversation(s), which should give you an objective measure of how much you are actually close to her. If you find the overall score too low after certain period, then you “revise” it to switch to a different one】
|
|
| [3] 标题： |
| 发表评论人：[游客]coolboy [2009-12-23 13:10:27] ip:173.67.2.* |
Through an “adaptation and learning” process, it is well said that “getting there is all the fun”. However, most people still hope to start the adaptation and learning process from a good initial condition. Hence, the natural question one wonders would be: can one or how can one put some efforts in the “stochastic optimization” of finding a good life partner before the marriage so that one would have a good initial condition to start the adaptation and learning process?
One plausible solution to this important and practical problem associated with the “stochastic optimization” was given in one of my previous comments and I think it is worth copying here again:
On Optimal Control
http://www.sciencenet.cn/m/user_content.aspx?id=209522
[3]Coolboy:【When there exists “uncertainty” in a system and one has to “fly with maximum speed in the direction of current position and B”, one had better to realize that it is not worthwhile spending great efforts on evaluating local gradient at each step accurately when the “current position” is still close to A and far away from B. If one evaluates the early stage gradient with little accuracy but great efficiency, and gradually increases its accuracy as the iteration solution approaches B, it is possible to avoid the notorious exponential growth problem for the multivariable cost functions.】
In brief, it says that an efficient and appropriate way of performing “stochastic optimization” is to gradually increase the efforts of searching at each step so that “THE optimal solution on expected or average value” can be achieved with the limited or a given resource. Implied in this kind of searching processes is that people often get more experienced when they repeat similar processes.
|
|
| [2] 标题： |
| 发表评论人：RongZheng [2009-12-11 9:17:56] |
| RongZheng将您的文章推送到学人亭,
|
|
| [1] 标题： |
| 发表评论人：[游客]coolboy [2009-12-10 11:39:46] ip:173.67.2.* |
I completely agree with professor Ho’s view on this matter. For those who want to learn more on the technical details about “adaptation and learning”, please read the discussions on the following two blog articles:
Go, get married, now! [coolboy]
http://blog.hjenglish.com/coolboy/archive/2005/06/28/77822.html
Is love powerful? [coolboy]
http://blog.hjenglish.com/coolboy/archive/2006/02/07/227419.html
|
|