Monday, July 28, 2014

Would Einstein buy into quantum physics as it is taught today?

Carver Mead, Cramer, De Broglie–Bohm all proved Einstein was not wrong, and we have no justification to think God play's dice. John Bell agreed and so did Schrodinger who remarked that his equation was deterministic on its very face.  His example of the cat was merely to show how ridiculous the Copenhagen interpretation was.  As mead said, we are not seeing a fuzzy quantum through clear glasses, but a clear quantum through fuzzy glasses.

In my view Einstein would never come around to accepting the notion of intrinsic uncertainty.  It is unscientific and unnecessary.  Anything can be proven with statistics.  For example when Einstein presented the mechanism of the laser to settle the issue of determinism, Bohr and Heisenberg swore it could never work until after it was demonstrated when they showed how certainty was caused by uncertainty.

Einstein would never in my view accept a timeless model of the universe as it was obvious to him that there was an immutable ordering of local event all observers agree on and those events, and only those events, determine the future deterministically.

Einstein's error was taking this notion of local causality one step too far or not far enough depending on your perspective.  In our participatory universe events are bi-causal, not cause and effect.  Causation is bi-local not non-local.

If you consider Einstein's own relativity and a near light speed observer traveling with a particle, as light speed is approached, distance and time go to zero making the event local to that observer, in the same place, at the same time, preserving Einstein's insistence on local causality despite the fact that the interaction occurred across light years from our perspective.  The fact remains that there is a viable reference frame where the event can be considered local causality and unless there is such a frame, there can be no spooky action.

With respect to many worlds, Einstein was only interested in our world.  The quantum exhibits universal general purpose logic where anything logically possible is possible.  It is "Law without law".  The strange thing is that it is possibilities that exhibit potential, not anything having substance.  We get "It from Bit".  As Smolin remarked, it is not many worlds, but many observers.  Entanglements are logical possibilities, not anything physical.  Lloyd remarked that a quantum computation takes place with perfect precision as if the rest of the universe was not there.  It is the available information that determines what can happen independent of any God's eye view of consistency.   As Feynman said the quantum can be understood as computing.  And it computes only with information (entanglements) available in a locality.

The ideal isolated worlds of Schrodinger and his cat would compute independently as if the other was not there until the box is opened.  Time would progress independently in the two worlds while the box was closed as there would be no mutual clocking of events between them.

I say we are teaching our children an archaic model of the quantum formed before Shannon's theory of communication and if we consider it to be an information universe there is nothing strange about it at all, it is quite logical, and the only way it makes sense for it to be as described by Lao-Tzu 2600 years before Planck and Einstein, the true founders of the quantum physics that is emerging picture of our inflationary information universe not bound to any substance.

I am certain Einstein would never buy into the popular notions of quantum physics today.

Jim

Tuesday, July 22, 2014

I got my hair, shirt, socks, pants and brain hacked at the Hackers On Planet Earth #hopex conference.

My wife likes the before picture much better.  My hacked pants and socks really got her.  I am not allowed in public with shorts anymore.

In 1964 I learned to use unit record equipment.   My first hacker encounter of the 3rd kind was in 78-79 on our research system doing field trials of group communication by computer researching collective intelligence at NJIT.  Starting design in '74 the computer science head proclaimed "nobody would ever waste the power of a computer doing electronic mail.".  We were way ahead of the curve, too far apparently.  The Electron Information Exchange System (EIES, pronounced eyes) was the internet to us, connecting the commercial packet switch networks, bitnet, arpanet, mailnet, amature radio packet meshnets.  It was all text based, no graphics, but it was amazing demonstrating powers and capabilities far beyond what we have ubiquitously on the internet today as remembered by many EIES users.  It was user driven to the extreme, users even automated themselves and shared capabilities easily.

One day "A. System Hacker" began participation in public conferences.  He explained how he connected to random addresses on the international commercial packet switching networks and was able to get in and take over an unused account using our all too helpful help system.  This was my first wake up call.  

Student jobs were not very plentiful at NJIT so we challenged students to hack into our security.  If they succeeded they got a job programming for EIES, their first job was plugging the hole.

There were a number of times I got sloppy and left holes exploited by hackers and lost a lot of work product and spent much time repairing the damage many times as a result.  In the late 90's I allowed the 4chan group on the original WikiWorld running as phpWiki.  I applauded their free expression philosophy unaware they attracted posters of the most extreme disturbing content.  After a WikiTrial there was a consensus to throw them out.  Well, they took down WikiWorld and forced a switch to MediaWiki.  WikiWorld never fully recovered.  WikiWorld was supposed to become an reincarnation of EIES on modern hardware and operating environments in the public domain.  It did not get very far, and hackers were at least partly to blame.

Since then, I have felt like I have never been rid of them, as if there was a war of good guys (ethical hackers) protecting me against the bad guys attacking me.  It is not like I have any hard evidence, just too many bizarre incidents to chalk them all up to coincidence.  But none has revealed themselves to me since "A. System Hacker".

Hackers have been in my world over 35 years but Hopex last weekend was the first time I entered their world.  I had a feeling I'd find both friends and enemies there.  I am a consultant at AT&T Shannon research labs doing biometrics, multi-factor authentication and security and thought this was a good opportunity to learn how then enemy works.  But, as a hobby I also hope to help resurrect WikiWorld and the EIES legacy, and I thought I might find some kindred spirits at Hopex due to decentralized nature and strong anonymous identities of WikiWorld who's protection might be necessary for its success.

When I arrived, I realized my business casual attire was a bit out of place.  Very quickly many of my preconceived notions were shattered.  It was no surprise that most were preoccupied with the evils of government surveillance and the loss of forth amendment rights.  And I was not surprised many considered any obstacle to be a challenge.  Tables with soldering irons were set up for hacking hardware.  One area was set up for picking locks.  Exhibitors offered challenges of breaking codes etc. for prizes.  Someone brought in a pay phone loaded with change.  Whoever could open it could keep the change.  I had expected I would find some radical anarchists there, and there were some, but they did not find sympathy there, they were scorned and ostracized.

What surprised me were the three dominant themes: self reliance, the conflict between privacy and transparency in society and the deep culture of social engineering.

I identified strongly with the self reliance culture.  My dad taught, if it ain't broke, don't fix it, but if is is broke, let's fix it!  This was a meetup of do it yourself enthusiasts.  Sessions on hacking cars and other devices was mostly geared toward self reliance and being able to fix things yourself or hacking them to give them greater functionality.  On the one hand, the sense of community was extremely socialistic but in support of individual freedom against the tyranny of the majority.  It was like a flashback to the sixties do your own thing that's groovy sort of thing rather than doing what the establishment wants you to do.

I was at first disturbed by people smoking joints right in front of the hotel.  It seemed they were begging for trouble.  Then I saw one person leaving the circle at a time saying they were going to look out, and another leaving when they came back, and an army of people not in the circle reporting possible trouble. And by the 2nd day I realized that they could socially engineer their way out of most any situation and were as safe as babies in their crib.  So I joined them :-p  There were orphaned bottles of top shelf booze at various tables for the taking.  That was a bit tempting but I have not had a drink in over 3.5 months and am done with alcohol.  The gave out free trip glasses and batteries.

There was deep concern over the conflict between privacy and transparency in society expressed in a number of sessions.  Ethical hacking involves making hard choices.  And the right thing to do is not always clear.  I had no interest in the whistle blowers sessions initially as I did not agree with their ethical choices.  However, there was significant content in most all of the talks and I cannot now so easily condemn their choices as is popular to do.  The potential imagined harm they do is portrayed as overriding any good by the media and so we are sucked into believing it.  They show how government cannot be trusted and despite the evidence they produce we stand behind the government and say the the government should be trusted and not second guessed and anyone who does not trust the untrustworthy government should be put in jail.  

They showed how without hacking anything the freedom of information act has been a wonderful tool in finding out the truth we do not want to hear of the policies and lies that victimize citizens.  I was shocked to see government investigator training that basically says to lie in prosecutions using "parallelization" creating a plausible (false) history supported by what evidence they have when there is insufficient evidence for what they believe happened.  All the time I thought is was just bad cops who twisted the truth, but now I see they are trained to do it in black and white, at least the feds are.   My brain was really hacked on this one.    We really do not want to know this stuff, but it is good that someone is looking.

This leads into social engineering which is really the flip side of how we are victimized by twisted truth where we do to them what they do to us, creating a plausible history that leads to a positive outcome so that we are not victimized, and can get what we want.

I was lamenting to a guy that is certain places in the hotel there was no hopex wifi available and it would be nice if we could use the hotel wifi that was restricted to people staying in the hotel.  He said, "its not a problem,  I searched twitter to find someone staying at this hotel, and then its a no brainer to socially engineer a way to get the room number.  All you need is the name and the room number to get on the wifi."

We can think of scam, sham or sting as social engineering, but there is an ethical side also in how a positive future can be derived from the creation of a plausible history.  It is not exactly honest, but can be achieved positively by just not telling the whole truth but selectively revealing information that is true, but leads the receiver to a false conclusion that that promotes a better outcome for everyone.  I suppose I am naive in that it took me 64 years to grok this but things in my life could be better if I did not always blurt out the naked truth.  What it really is about is fighting fire with fire or being defeated.  The ethics are dubious and it is too easy to overdo it where it may not be necessary but often it may be the right thing to do.

Ralph Waldo Emerson said that a mind once expanded can never go back to its original dimension.  This was an mind expanding experience for me.  I got my hair, shirt, socks, pants and brain hacked at Hopex, and I connected with folks very interested in WikiWorld.  It was a great experience.

Jim

Sunday, December 5, 2010

How are capitalists bad?

On the history channel, the presidents series, Herbert Hoover was quoted as saying the only thing wrong with capitalism is capitalists. This got me thinking about what was wrong with capitalists and how what is wrong with the economy might be attributed to bad capitalists.

Capitalists are market participants seeking to maximize value. Their accumulated value represent capital they apply to accumulating more value. It is in the interest of capitalists that there is a thriving market. We are all capitalists since nobody want to minimize value. Rather than bankruptcy they seek to improve their lot.

The hallmark of every golden age is a thriving economy. Capitalism has dominated all successful economies. There is no proven alternative. Most everyone benefits from a thriving economy.. But economies often have downturns or contractions. Often the poor suffer while the rich languish. The few drain the wealth of the many. Capitalism is generally good in a growing economy, and woefully bad in a contracting one.

The economy has an evolutionary nature, and is stimulated or suppressed in various ways. The wealth of the general population provides the markets that can be capitalized on. Human enterprise produces value in the markets. Certainly capitalists that produce value in the market are good capitalists.

However, there is another side to capitalist activity, cornered markets, exploited resources, and vulture capitalism that profits from the destruction of value.

Those who cherish the promise of capitalism to enable humanity to thrive such that their are strong markets with free enterprise, but then there are capitalists who are willing to kill the golden goose, and hurt the market they are playing in.

The strength of economy can be improved or detrimented by the capitalist. Detrimental activity is at the expense of many for the benefit of a few. It is self defeating activity that can lead to economic downwall as overall value is reduced. Such capitalism is infective in sustaining growth. It capitalizes on and depletes scarce resources.

The world is rich in resources and opportunities, capitalizing on abundance is how civilization can thrive and we can populate the galaxy and beyond. Capitalizing on scarcity will lead us down the road of civilizations that have perished to go the way of the dinosaurs.

Effective

Sunday, April 25, 2010

Email Identity Theft.

My gmail account was hijacked apparently. Many of the addresses on my address list received spam from me at 9:00 am EST 4/24/2010. This makes the issue of email identity theft personal for me. It is an issue that needs to be addressed before it effects you.

I've changed passwords, etc. But once they have an email list, it gets shared among spammers and I cannot stop them from using the email addresses forever. Damn.

What most likely happened is that a sight was hacked where i registered with the same password as gmail. Shame on me Or, I shared my address list on a social networking site that was hacked, shame on them.

If your email was in my address book beware of links presumably sent by me but not signed "Jim". They are not from me really, they are spoofed. Sorry folks.

Now I understand why emails clearly not spam from some members are flagged as spam in groups I moderate, and why email from some people often gets blocked as spam. I had wondered if they had been spammers in their past that they were being punished now. It makes more sense that they had been victims of email identity theft as I now am.

I ban members from my groups who enter spam, and report spam when I receive it. Unwittingly I have been victimizing the victims of email identity theft. As I judged, now I am judged. If the spammers send a lot of email from my account my electronic world will get smaller and smaller. The victims of email identity theft will become isolated and not contribute what they otherwise could to society.

I am still trying to notify everyone in my email address book, but there are too many addresses and gmail blocks me from sending email for 24 hours when I hit the limit. I can't even defend myself.

It is impossible for any national government to control email identity theft. China. a most repressive nation is where must of the spam originates or is relayed. It is a good thing the internet is an open bottom up protocol. Only name services are controlled centrally. The problem is not the freedom of the internet, the problem is that 99.99% of us are not using digitally signed email.

This will not change until email clients people use to send email validate and encourage using digital signatures, and that will not happen until people choose to only use clients that do. A digital signature uses cryptographic techniques to provide verifiable proof of authorship. This has not happened because standards need to be accepted to accept digital signatures in emails that are backwards compatible with the billions of legacy clients. There is no accepted standard means of including a digital signature in a plain text unencrypted ordinary email. Any suggestions?

The problem is the transition. If there was a standard way to digitally sign a plain text email in the header or the body, any email client that already had your public key, or could get it, could flag spoofed emails as not really from you. Email programs that did not support the signitures would still work fine but with no identity check. Email providers and mailing list and forwarding programs could have keys for their email addresses and support this

The use of a service, one time, to create and serve your keys, and an email client that supports it, is all that is needed.

If one big email provider, like google, supported this, I think it would give them a market advantage, and every client would follow along. The threat of email identity theft and spam generally would be greatly reduced.

Jim

Links to more info:
http://mail.google.com/support/bin/answer.py?hl=en&answer=50200
http://answers.google.com/answers/threadview/id/14096.html
http://www.scamdex.com/
http://en.wikipedia.org/wiki/Digital_signature
http://en.wikipedia.org/wiki/Pretty_Good_Privacy

Wednesday, April 14, 2010

Remembering GrampScarver


Remembering GrampScarver

[My memorial service testimonial plus added missing sentiments expressed by others, afterthoughts, etc.]
 
The financial crises has left me out of regular employment and I have been thus blessed with the wonderful opportunity to be with mom and dad through our challenges of the last year.  Dad lived well and died well.  It was almost a year ago when he announced his intention to die.  He told us all he had lived a good life and done everything he wanted to do.  Near the end, he blamed me that he lived too long for taking too good care of him.  I feel, he was not going anywhere until he was convinced my mom would be okay.

In the end he said he was making no more decisions.  He was delegating to me the hard job of making all the decisions for him.  I said "dont be silly, taking care of you and mom is easy and I am happy to do it.  It is a blessing for us all that I can be here now."  I am beginning to see the gravity of the responsibility I accepted that is only increased in his death.  To do his will, I must remember all that he taught us by his words and by how he lived.

He was the independent individual in many ways.  He was always ahead of his time technologically.  We were years ahead of the croud in having a rotary lawn mower, video game, microwave oven, digital watch, calculator, computer, etc.  He never bought what was the current fad, he got what he wanted that nobody else had. 

You would not call him gregarious as he shied from social activities. He seemed a bit cold and distant on first impression.  Yet everyone liked him and he was most always pleasant, helpful, knowledgeable, and intelligent.  Everyone respected him.  Many sought his council.  He was an avid reader. 

Delegate everything.  His business ran like clockwork without the necessity of his involvement.  Instead he spent time learning, strategic planning, and working alongside every employee every year.  His employees, for the most part, loved him, and too good care of business for him.

Every problem is an opportunity, the challenge is finding it.  There are no problems, only solutions.

When I wrapped the new family car around a tree, and said, "Gee dad I'm sorry." he said, "No problem, things are not important, people are."  That material things are not important to me is a great gift he has given me.  I do not suffer, as many do, over material things.  Money was never a problem, so it was not important.  He was frugal though he did not need to be, and never succumbed to consumerism.

I grew up thinking he knew everything.  Sitting on his knee he taught me everything from psychology and political theory to quantum and space time physics.  It was not until I was 35 that I asked him a question and he was not familiar with the prevalent views and issues and said simply, I don't know, leaving me in a state of shock.

Whenever we said, "it was an accident", he would interject, "There is no such an accident, everything happens for a reason.  "

Only you are responsible for what happens to you.

He had a  most unconventional wisdom and was a master of reverse psychology.  He gave old adages a Twainian twist that was uniquely Franklin.

Spend your money, don't save it.  Any idiot can put money in the bank, the real challenge is to spend it well.

Never set an attainable goal for your life, since once you attain it, your purpose in life is lost.

If you are only willing to try 99 times, don't bother trying

If you try to do two things, you will not succeed at neither.  If you do one thing, nothing is impossible.

The most intelligent person is the one who makes all possible mistakes first.

It is better to act, and be wrong, than to not act.

When he attained his financial goals in business he no longer had any interest in business and began a new technical career from the bottom.  He told us to never set an attainable goal for our life, since once attained, purpose in life would be lost.

Never concern yourself with what you cannot do, only consideration of what you can do is worth while.  If you can't change it, don't worry about it, just stay out of its way.

Do not try to please other people, all that really matters is that you please yourself.

Don't care about what other people think, care what you think.

Don't do what I tell you to do, do what you believe is the right thing to do for you.

Don't do anything you don't want to do unless there is no reasonable alternative.  It is doing what you like doing that will be most fruitful.

In words he taught us to be selfish.  He spoiled us.  He let us do almost anything we wanted to do.  He sometimes expressed what he thought the negative consequences might be, but he did not try to control us.  All my friends wished they had my parents.  It was not called love, but not controlling us was the expression of his love.  He told us we were free to face the consequences of anything we wanted to try, but was there to pick us up when we fell down.  There could be no greater love.

By his example, in deeds he exuded selflessness, deep respect for the feelings and wishes of others, and encorgagement of others to be all they can be.  He taught us how to love.  Many have expressed eternal graditude toward dad for encoraging them to change careers, or otherwize fulfill their dreams rather than compromising who they could be and accepting less from life than what they really wanted.

When a group of NJIT colleagues and I incorporated our company, Xanthus, we used my dad's attourney.  who asked us if we thought we could conduct business with the same degree of integrity as my dad.  After a pause, I said confidently, YES.  Hw lowered his eyes and said softly, "I don't think so."

My sons could not come here today but are with us in spirit.  Jeff said it well, this should not be about death, but about life, and that dad lived and died as we all should.  Their mom put it best, that he has always supported us in so many ways, and most significantly, he taught us how to think

In some ways he had a contrary nature, though he respected others beliefs, he often expressed opposing viewpont in a Socratic fashion.  Much interaction with dad was argumentative,  Jeff said he argued with him fruitfully for almost 35 years, for me, it is nearly 60 years.

In his final days, a new sweetness of character, formally reserved for my mother, emanated from him.  He complimented the health aids and nurces profusely.  With happy eyes he would welcome me with, you are a good boy jimmy, I love you.  The last days he spoke, I asked how he was doing today, and with great warmth he replied, wonderful.

Although many remember him as a very spiritual person, who influenced their beliefs profoundly, at the same time, he believed in nothing himself. He did not deny the possibility of anything, and was always searching for answers.   But he held virtually no doctrine, save the wisdom of Socrates, which he posted prominently on the refrigerator,  "The only true wisdom is the knowledge you know nothing.".  He challenged the objectivity of any belief in God.  In the end, however, he found a notion of God he was comfortable with, the power of truth in a complex world that is beyond any possibility of human comprehensioon.

In his last days he asked my daugher if he was a bad man.  She said he was good.  He asked what made a person good.  She said "the capacity to love".  He asked what about the capacity to live?

And today we are here to celibrate the wonderful life he led, and take on the responsibity of continuing his legacy of thingkng out of the box, and living his mandate that we live our lives our own way as he did it his way, wonderfully so.

Mom said it best.  "He was a wonderful husband, father and friend.  He was exceptional in every way."

My son Jeff said, "everyone should live as he did."  He taught us how to think and act, and delegated his thinking and acting to us.  In the end I think he believed his legacy would be fullfilled though us.  I wish I was so confident.  I cant imagine anyone filling his shoes.  Together we can only try. 

His goodness depends on our capacity to do as he did.  For him love was action, not words or feeling, and his love was expressed in how he lived.  His religion is not one of belief, it is one of right action being true to oneself, while enabling others to be true to themselves.  For me this is not different from loving God, and loving one another as he loved us.  Like Jesus, grampscarver is a great example who's time of realization is yet to come as we are only beginning to to learn how to emulate him. 

His example demonstrates that to be selfless, we must first be selfish, or we have no self to share with others.  His life is a great gift for all of us, and gives us a hope that in remembering him, and following his example, we might live and die well, as he did.

JimScarver

Wednesday, March 25, 2009

Monetary liberation - the cloud monetary system

I am in the process of envisioning an internet cloud market exchange system, bartering essential value and central power currencies, independently based on personal trust and intrinsic value.

Ron Paul is correct in stating centralized money amounts to price fixing, and is contrary to a free market. The taboo against against price fixing is well founded in the literature and history. Monopoly stymies the free market by restricting evolution of the market. In evolutionary game theory, Pavlov, or the learning agent, defeats the bully in the evolutionary game. But this assumes a level playing field. The playing field is not level when one group controls the money supply and thus has a blank check to dominate all other players.

YouTube - Ron Paul "The Federal Reserve Is the Culprit!

http://www.youtube.com/watch?v=MpQlB3f9Al4

I do not agree with Ron's isolationism and see a gold standard as counterproductive, going from the frying pan into the fire, as the money powers already control the gold market. But nobody is perfect and only one republican and one democrat in the US congress support monetary reform thus far. That we must put the power of money in the hands of the people is the essential common thread.

YouTube - Dennis Kucinich States His Intention To Put The Federal Reserve Under Government Control


http://www.youtube.com/watch?v=-r_-QRKyu6g

There is a pyramid of power, dominated by money, and thus the the gamblers with the most potential money. Essential value, belongs to production, rather than gambling. It is imperative, that money largely represents essential value, in a fair contract, to have real value. The international currencies have no essential value and ought not be the primary means of exchange without essential monetary reform. I promote minimalistic monetary reform to eliminart public debt and money as debt in http://MonetaryReformAct.org

Steve Keen’s DebtWatch No 31 February 2009: “The Roving Cavaliers of Credit”

http://www.debtdeflation.com/blogs/2009/01/31/therovingcavaliersofcredit/
reveals many of the essential facts I have yet to clearly express on http://MonetaryReformAct.org but the whole picture is still incomplete.

See also:

URL: http://www.rollingstone.com/politics/story/26793903/the_big_takeover/print

The Big Takeover, The global economic crisis isn't about money - it's about power. How Wall Street insiders are using the bailout to stage a revolution, MATT TAIBBI, Posted Mar 19, 2009 12:49 PM

Grass roots money requires a trusted exchange mechanism. When I was laid off as an information technologist doing brokerage services for Banc of America in the merger with Merril Lynch, the last independent major investment firm not previously owned by an international bank, my friends and their anonymous cohorts at http://WeAreGreat.org suggested I ought do my own stock exchange. A Barclays prime brokerage and BofA I have been in the belly of the beast. I have the necessary experience to do my own exchange. But the idea is ridiculous, but also haunting. I have 25 years of research experience in collaborative systems as well.

But I am an idiot savant, socially inept, lexdysic. It seems I can't know how to do exchange others trust though I am competent in my work and deliver information systems that work. So I leave the challenge to others. I will indeed develop my own exchange for myself, by leave it to others to subjectively trust whatever they trust. Barter may be facilitated according to trust. The system can work immediately as many people mistakingly trust national currencies. This gives alternative currencies the opportunity to gain greater trust.

The infrastructure necessasary, which has been my quest since 1974, now exists on the web.
Re: eies3/web3 etc. http://tech.groups.yahoo.com/group/InfoPhysics/message/6544
The EIES principle of supporting the individuals privacy and power, enabling the individual to facilitate a collective intelligence, is no longer a pipe dream.

Money is a form of trust. The way to facilitate money is to facilitate trust. Not some abstract notion of trust, but trust as it actually exists independently on the cloud. Essential value is due only to trust. Each of us ought to be able to use only money we trust represents essential value. The actual future value is always uncertain but we are not truly free if we are not free to use that which we trust.

I favor decentralized money and free enterprise. I am not partial to capitalism except in the sense of decentralized global value creation. I am not a pure capitalist in spirit and lean toward free and decentralized communal living. In my view, money should not be necessary, except where it is a necessary evil. Free enterprise in a fair market simply maximize our human possibilities.

By "fair market" I only mean not giving control of the money supply to the owners of the private central banks or other elite group. There is no free market if one group has carta blanca, a fair market is a free market. Capitalism is bound to collapse unless we have a free market without centralized control.

I do think we have a responsibility in our collective action to use the power of our pocketbook to encourage market diversity where resources are squandered, monopolies rule or where markets are being displaced. We can thus drive the system toward sustainable growth with minimal discontinuities and suffering.

Gambling is distinct from investing as investing potentially creates essential value, while gambling only seeks to take from others without returning value.

In the present crises no essential value has been lost, instead the money supply created by gambling debt has dissolved making money scarce and causing a collapse. We need a stable currency immune to the activities of gamblers.

I hope to readily assimilate with those of like mind in any respect. What could be better than dreaming and becoming, thus promoting into existence, that which which might be, together?

"History records that the money changers have used every form of abuse, intrigue, deceit, and violent means possible to maintain their control over governments by controlling money and its issuance." -James Madison

The pyramid of power includes central bank owners at the top, multinational corporations, the military-industrial complex, government political machinery, religions, media owners, mafias, and finally the people. Each is has power as allowed by the powers above it, who ultimately rule our destiny and largely determine our success in changing our lives and our world at any level.

"The issue which has swept down the Centuries, and must be fought - sooner or later -is the people versus the banks...Power tends to corrupt, and absolute power corrupt absolutely..." , Lord Acton

Whatever other action we might take will be futile unless we each refuse to use money controlled by some elite. To that end we should use alternate currencies or vouchers, alternate financing, and promote monetary reform.

"Bankers own the earth. Take it away from them, but leave them the power to create money and control credit, and with a flick of a pen they will create enough to buy it back." - Sir Josiah Stamp, former Director, Bank of England

At issue is how we can be effective personally in making alternatives practical.

Jim
http://MonetaryReformAct.org
Jim
http://InformationPhysics.com
http://MonetaryReformAct.org

Tuesday, March 24, 2009

Troubleshooting excessive linux iowait time

I hate when a unix system gets really slow but is doing almost nothing. Typically what happens is the load sky rockets, idle time goes to zero, and cpu system and user time usage is very low.

The vmstat command is great for monitoring the situation.

vmstat 30

procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
6 33 1131712 9400 5740 103116 2017 5978 3022 5995 657 855 4 6 0 90
0 31 1172848 106668 2664 41288 3792 4598 4496 5061 726 1094 25 8 0 67
1 31 1146468 8984 1088 19060 2559 674 3034 882 559 642 19 9 0 71
1 30 1238520 15116 2008 22084 2362 4689 2517 4707 712 878 3 6 0 91
1 26 1345528 12440 4756 55188 3861 6525 4744 6729 775 1141 5 9 0 87
1 27 1439400 20520 4952 61852 2641 6133 2892 6889 734 914 6 7 0 87
2 22 1505548 291744 988 17412 3973 5662 4295 5684 802 1194 16 9 0 75



The vmstat output above shows a system spending most of its time in iowaits with a little more than half of the activity being swapping shared memory.

The machine has 1GB ram and up to 1.5GB swapped in above output. Swapping might be eliminated by adding another gig of memory to the machine. This might alleviate the problem.

You can check which processes are using the most memory with the ps command.


ps aux|sort -n +3|tail -10
apache 27288 0.6 2.5 198840 25684 ? D 08:28 0:03 /usr/sbin/httpd
apache 27923 1.7 4.0 197796 41512 ? S 08:34 0:02 /usr/sbin/httpd
apache 27874 1.1 5.3 214640 54096 ? S 08:33 0:02 /usr/sbin/httpd
apache 27849 0.8 5.6 214648 57504 ? S 08:33 0:01 /usr/sbin/httpd
apache 27852 1.3 5.6 214892 57712 ? S 08:33 0:02 /usr/sbin/httpd
apache 27887 4.1 6.2 273356 63160 ? D 08:33 0:09 /usr/sbin/httpd
apache 27927 7.1 10.9 273356 111236 ? D 08:34 0:09 /usr/sbin/httpd
apache 27890 3.3 11.4 273608 116184 ? S 08:33 0:07 /usr/sbin/httpd
apache 27929 4.0 13.0 290040 133308 ? R 08:34 0:05 /usr/sbin/httpd
apache 27334 2.2 16.7 328072 170808 ? D 08:28 0:11 /usr/sbin/httpd


The above command sorts processes by percent of memory usage (4th col) and shows the ten biggest memory pigs. In this case, the apache httpd processes listed account for about 70% of available memory.

If you still experience high iowait time when the system is not swapping heavily you need to investigate which processes are doing the io. The following scrip puts together vmstat, block_dump, and ps, to show system activity and the largest io consumers during that activity.

bin/whowait
/etc/init.d/syslog stop
echo 1 > /proc/sys/vm/block_dump
uptime
vmstat 5 6
uptime
dmesg | egrep "READ|WRITE|dirtied" | egrep -o '([a-zA-Z]*)' | sort | uniq -c | sort -rn | head
echo 0 > /proc/sys/vm/block_dump
/etc/init.d/syslog start
echo Top memory hogs:
ps aux|sort -n +3|tail

Outputting:

Shutting down kernel logger: [ OK ]
Shutting down system logger: [ OK ]
09:41:06 up 5 days, 20:48, 1 user, load average: 3.90, 3.58, 2.72
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 2 815844 201800 2052 72560 632 831 1141 1802 64 42 18 5 45 32
0 1 815196 277960 2424 72784 1221 0 1298 396 452 624 6 5 1 89
0 0 815196 275816 2560 74004 3788 0 3931 5306 589 753 16 10 6 67
0 0 815196 275816 2560 74004 177 0 442 114 304 175 5 1 72 22
0 0 815196 275716 2564 74004 19 0 19 77 272 98 0 0 98 1
0 0 815196 275344 2608 74136 18 0 25 33 267 114 0 0 96 4
0 0 815196 275344 2616 74136 32 0 32 7 259 93 0 0 99 0
09:41:32 up 5 days, 20:48, 1 user, load average: 2.74, 3.33, 2.66
1074 httpd
782 named
567 kjournald
177 pdflush
130 qmail
130 mysqld
45 java
9 miniserv
6 executable
4 bash
Starting system logger: [ OK ]
Starting kernel logger: [ OK ]
Top memory hogs:
apache 31245 1.9 2.7 185932 28100 ? S 09:22 0:21 /usr/sbin/httpd
apache 32073 0.4 3.1 190620 32364 ? S 09:36 0:01 /usr/sbin/httpd
apache 32106 1.3 3.1 190608 32176 ? S 09:37 0:03 /usr/sbin/httpd
apache 30959 2.4 3.2 190644 32816 ? S 09:20 0:31 /usr/sbin/httpd
apache 31225 3.0 3.2 191024 33144 ? S 09:22 0:34 /usr/sbin/httpd
apache 30955 1.8 4.0 198840 40964 ? S 09:20 0:24 /usr/sbin/httpd
apache 31982 4.0 5.1 210604 52704 ? S 09:35 0:13 /usr/sbin/httpd
apache 31983 3.1 5.3 212300 54220 ? S 09:35 0:10 /usr/sbin/httpd
apache 31338 1.7 10.8 269196 110512 ? S 09:23 0:19 /usr/sbin/httpd
apache 31924 1.9 10.9 269176 111220 ? S 09:32 0:10 /usr/sbin/httpd

Here the load is not too high and a sample during higher load would be more meaningful.

We may also want to see exactly what files are opened by blocked processes as follows.

for i in `ps -eo pid,user,wchan=WIDE-WCHAN-COLUMN -o s,cmd | awk '$4 ~ /D/ {print $1}'`; do echo "----------"$i"----------"; lsof -p $i; done|more

In the above system the issue to be resolved turn out to be the longevity and memory use of apache processes. This can be achieved examining the apache logs. If you want to see what is accessed recently and have many separate log files to check, the following command is useful substituting the wildcard path to all the access log or error logs on your server.

tail -1 `ls -tr /home/httpd/vhosts/*/statistics/logs/access_log|tail -5`

==> /home/httpd/vhosts/ippsa.org/statistics/logs/access_log <== 68.254.73.3 - - [24/Mar/2009:13:07:57 -0400] "GET /html/index.php?name=News&catid=&topic=12 HTTP/1.1" 200 11618 "http://www.ippsa.org/html/modules.php?op=modload&name=Topics&file=index" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7"
==> /home/httpd/vhosts/eies.org/statistics/logs/access_log <== 74.247.231.3 - - [24/Mar/2009:13:10:21 -0400] "GET /cgi-bin/ip/herrmann.eies.org HTTP/1.1" 200 184 "-" "curl/7.12.1 (i686-redhat-linux-gnu) libcurl/7.12.1 OpenSSL/0.9.7a zlib/1.2.1.2 libidn/0.5.6"
==> /home/httpd/vhosts/wikiworld.com/statistics/logs/access_log <== 68.151.117.181 - - [24/Mar/2009:13:18:25 -0400] "GET /wiki/index.php/ClassicalLogic HTTP/1.1" 200 3497 "http://www.google.ca/search?hl=en&client=firefox-a&rls=org.mozilla%3Aen-GB%3Aofficial&hs=2X&q=platonic+reasoning&btnG=Search&meta=" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 (.NET CLR 3.5.30729)"
==> /home/httpd/vhosts/waldorfwithoutwalls.com/statistics/logs/access_log <== 24.5.48.111 - - [24/Mar/2009:13:18:26 -0400] "GET /i/h1articles.gif HTTP/1.1" 200 3085 "http://www.waldorfwithoutwalls.com/articles/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7"
==> /home/httpd/vhosts/whitescarver.com/statistics/logs/access_log <== 64.66.192.62 - - [24/Mar/2009:13:18:23 -0400] "POST /gallery/Family-Photo-Phun/IMG_1486?full=0 HTTP/1.1" 200 708383 "http://www.whitescarver.com/gallery/Family-Photo-Phun/IMG_1486" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"


This shows the last entry in the five most recently updated logs and can easily be adjusted to show more or less. After the fact you can examine each log for the time the problem ocurred to see what was accessed.

One problem that needed to be corrected was spiders (web robots) accessing time consuming functions that need not be indexed
and should not be spidered. Entries were added to the robots.txt file in the document root of site to exclude most of the robots from these pages.

It may not be clear which log entries are problematical. To find log entries for specific processes the process id must be recorded in the log. You will most likely need to add %P to the LogFormat in your apache configuration, /etc/httpd/conf/httpd.conf etc.

LogFormat "%h %l %u %t \"%r\" %>s %b %P" common

To be continued...