• Foreigners are generally not welcome in most restaurants, onsens, entertainment establishments, etc. unless accompanied by a Japanese native
• Japanese will typically avoid sitting next to a foreigner in public places - there are innumerable stories of how Japanese would rather stand than sit next to a foreigner or if a gaijin has the nerve to sit down on an empty seat next to a Japanese, they will either move away or stand up
• In jobs advertised in even English language newspapers, most of the time it will be clearly mentioned that foreigners need not apply.  In Japanese language newspapers, it is understood that foreigners will simply not apply for jobs
• For those jobs where foreigners are hired sometimes - for instance as language teachers - the discrimination is evident.  The salaries are lower than that for a Japanese in similar jobs, expectations are higher, firing is easy, and regardless of the importance of the job, the foreigners are never part of the inner circle - which means that decisions are taken without them and they are simply expected to execute them

Japan has a very strong history of discrimination as a part of its culture and we are talking about discrimination of its own citizens:

• Even Japanese citizens raised overseas are regularly discriminated.  Those that have spent just a few years or have absorbed some non-Japanese customs or cultural attributes are discriminated on a regular basis
• Children from parents with a non-Japanese partner are doomed in Japan
• Folks from the countryside never make it to the top
• If even one member of the family ends up in an embarrassing situation (crime, poor education, low end jobs, etc.), the other family members have to share the discrimination for generations
• The physically disabled and mentally challenged are discriminated to a point that Japan pretends that they do not exist.  It is one of the un-friendliest countries in the developed world for physically handicapped people - many public buildings and means of transport have no provisions for disabled people.  Japanese companies often prefer to pay an annual penalty for not hiring disabled people.  Lepers are treated worse than animals in Japan
• The condition of women, though improving, is clearly a result of discrimination over centuries

Japan is a highly homogeneous society and any differences from the norm is frowned upon.  It really matters little whether you are a foreigner or not.  Of course, for the same situation, a foreigner is likely to feel a higher degree of discrimination due to the preconceived perceptions that "foreigners just don't know the Japanese way" but a Japanese is not likely to spared either.