Fabulous IT

This is clearly not a continuation of the UP4edu post... I am still working on that, bogged down in tangential projects. One such tangent is an exploration of language, medium, and technology choice. It started with a reference to The Alphabet vs. The Goddess in no fewer than two Agile books. I am wanting to say one was in a Cockburn book and the other was Wicked Problems Righteous Solutions, but sadly I neglected to note either in my Tomboy notes.

This resonated with some of the left-brain-right-brain reading I had been doing in another peripheral scan. There are clear connections between these topics and: learning styles, accessibility, inversion of control, and information mapping. I am hoping to do a satisfactory, though admittedly never complete, sweep of this whole tangent soon so i can get back to UP4edu. A light and sufficient exploration if you will.

I propose that the predecessor to rich media was conceived in the moment the first illuminated document was drafted. This likely predated the Bible. Any association of pure text, and pure imagery will do. There are three distinct combinations of "pure text" and "pure imagery": iconographs, illuminated or illustrated text, and assemblage. Ranked in that order, any combinations of these three would likely fall under the guide of the highest ranked component. Iconographs are technically a form of pure text and barely count, so primarily I am concerned with the latter two.

Iconographs are very close to my heart, but their role in rich media is only cursorily different than elements of pure text or pure imagery. Their unique status of being both doesn't lend anything profound as all pure text as artistic elements and all pure images have communicative value.

Love and Joy About Letters (Shahn) is a delightful exploration of the interplay between illustration and assemblage. There is in fact a spectrum between these two facets of textual images. A spectrum exists between each and iconographs, but it is a much less profound spectrum.

Images and text both invite interpretation, but in different ways. Images overpower text, and yet humans gravitate to the safety of words. Images are volatile. Words seem safe, and they can certainly be crafted with great precision to be clear in intent. In the end, rich media brings us back to the realization that these two very different forms of communication have weaknesses.

Text is merely mundane iconography for an intangible, mutable, and universal unit. Text can be translated. The medium is weak and arbitrary, but there is an underlying reference to pure and raw information.

Images can only be experienced through the visual sense. Translations of images into text or any other experienceable sense are paltry in comparison. Here the experience is pure and raw, and the medium is of the essence.

Shahn's journey is religious and worldly, a man's exploration of the things that have happened and are happening to him as seen through the lens of one that believes in the divine. There is innate integration of spirituality and the world, just as there innate integration of image and word.

Shlain, in the Alphabet Versus the Goddess, asserts quite strongly the effect of language and language medium on our sense of religion. This hints at a pervasive inversion of control, that is to say a foundational technology being an influencing force in the direction of evolution of nearly all societies. Both spoken language and written language would have a hand in the process. Art too would be implicated. The cyclical effect visual art and literature has played is rarely questioned in modern dialogs on the matter. The driving undercurrent of the written and verbal forms is however a topic that is much more commonly held below the socially conscious surface.

Admittedly, I am only half way through the Alphabet Versus the Goddess. It seems herculean in effort to work though, much like Cosmos (Sagan).

Illuminated texts present a problem for Shlain's portrait of the early Jewish divine. To invoke a modern meme, clearly at some point someone was "doing it wrong" and they have expounded that error a thousand fold sense. If the Ten Commandments really do counterindicate images in the way Shlain suggests, illuminated texts, and the attempt to visualize the divine is entirely misguided.

One need only look to inspirational pieces such as The Saint John's Bible (Sink, The Art of The Saint John's Bible) to see this dilemma. It mixes divine word with an extra layer of interpretation. Time will only tell if this is a true modern rapprochement, or a reincarnation of Babel.

Gender is much ado in the first half of Shlain's book. In a more modern context it no longer seems relevant what gender God may be, if it is even appropriate to assign gender to the divine. The latter has been my assertion for the entirety of my adult life. Such would be the ideal, assuming the foundations of our language are not somehow subversively continuing to deter movement towards a society that embraces diversity and equality.

It seems comical to many (Introducing the Book) to consider books as a technology, much less the letters of our alphabet. Color television is more obviously a technology to some than color paints.

The elements of visual communication include (cobbled together from personal experience; Visual Language, Bonnici; and A Visual Language, Cohen & Anderson): color, line, direction, shape, boundary, contrast, texture, proportion, relative placement, and complexity to name just a few broad aspect categories all affect the cycle of message, expression, and interpretation. The mindful application of these aspects are a technology.

Single images, simple or complex, much like single letters, words, and sentences convey ideas and impressions. It may be a complex idea, but rarely will a single image convey narrative consistently. A single image may induce narrative through recollection or imagination, but this relies on contextual interplay in the viewer. Sense of direction and relative placement can create narrative, but more often the end result in cognition is more of a compound image than a single image.

Sequential images (Sequential Images, Wigan) set the stage for narrative communication through image. A composite image, as differentiated from a "single" image, may have the capacity to express narrative if the order of sequence is clear. Ambiguous order can create narrative, but in an unpredictable way. Inevitable order creates more cohesion. Optional images, and interactivity as a driving force in image sequence and content create and entirely different forms of narrative.

Words too can create narrative, seemingly much more effortlessly for the person conveying information. In contrast, images are typically more effortless for the person interpreting information. This is perhaps one reason combinations of the two are increasingly popular, there is a strike in balance between cost to produce and cost to consume. Broadening the audience is another likely reason. Lexical differentiation is possible and powerful, but visual differentiation is more immediately striking. A two fold attack creates powerful rhetorical reinforcement.

The consumption of images is an "all at once" practice. Unlike reading, where the primary mode is linear scanning, taking in an image starts broad. Color and boundary perception are a strong suits for instant recognition, and it is a large factor in animation and creating data visualizations. Shape too can be instantly recognizable, but is plays a second string to color and is quickly trumped in visual processing priorities. Shape, in ways it differs from Boundary (the two are inexorably linked) seems to require more coordination with between the two sides of the brain. Considerations along these lines are important when sequential images come into play. The speed at which these images can be processed for picking out spacial arrangements of color and shape simultaneously tend to favor color.

It would seem desirable to select language technology based on some ideal set of criteria. These might include: easy to learn, fast to read, and highly memorable. Other considerations are also important. The level of expression and creative choice could easily have an impact on the culture built upon the language. Adaptability, parity, and compact representation are all factors in how effectively and efficiently language can be used. Written language is clearly visual, but how well it uses the visual senses can be brought to question.

One could argue that the selection of language form is driven by human tendency, just as the assumption that technology adoption is driven intentionally. Assuming a natural selection of such things might seem reasonable, but the means of selection in this case are the very things that have advanced societies through the ages:  violence, material greed, and stratification. Clearly, these are not the lofty ideals upon which I would like to see society and subsequently technology built upon.

If we don't understand the true reason one technology has won out over another, assuming a natural selection is folly. More over and all things being equal, natural selection does not favor the best overall or long-term solution, it favors the immediate best.

Human nature has great potential to both correct error against overwhelming odds and compound error despite clear consequence. One can only hope that is there is a surreptitious effect in our choice of language, that it can be mitigated.

Language bias, an unnatural allegiance to language can be observed in individuals. Possibly innocuous in most cases, it can in others be as dangerous as any form of discrimination. Admittedly, we are very much indebted to societal progress due to language. When this translates into a conscious or subconscious valuation of particular language however, there is a problem.

I first became aware of the existence of this when discussing the speech challenges a friend's son was having. He was perfectly bright, but verbally behind what might be considered normal progress. I inquired if they had looked into exploring sign-language as an auxiliary means of communication. The child had no diagnosable hearing challenges, but had mind-to-severe speech impediment. My friend was optimistic on this possibility until her finance chimed in that he had heard that going this route would be problematic. Reliance on sign-language could become a "crutch" that the child would too readily fall back onto rather than "pushing through" with progress on speech.

This alone was not too alarming, but subsequent conversations with other people on the popular subject of immigrants and language resonated. It further became clear what the issue revolved around when reading about Deaf culture and the political issues around hearing impairments and medical devices (such as cochlear implants) that supposedly enable people to live "more normal lives". Friends who I would have never in a million years thought of as discriminatory didn't seems to understand that someone might legitimately desire to not have the implants. They uncategorically saw parents refusing the highly invasive surgery on their infants to be considered a form of neglect.

Clearly, this is a case where in the minds of some, proficiency with a specific language and being "like other people" with the same basic capabilities is equated with judgment of human value. There is a base line definition of what is important to be considered human. Hearing is clearly on some people's list, even if its inclusion isn't conscious. It was especially interesting where some individuals did not hold consistent views: they felt forcing immigrants to know English was ok, but forcing deaf individuals to accept "hearing values" was not, or vice-versa. It is similarly interesting that parents of Autistic children seem much more willing to embrace modification to language than hearing parents of deaf children.

While I will go so far as to say that this can be attributed to the adage: "Best is the enemy of the better", that might be interpreted by some as heresy. Parents that are given reason to believe there may be a "cure" are likely to fixate, reasonably if not unrealistically so, on the proposition without any real consideration for the genuine nature of such a cure. Some doctors and companies hype up the level to which cochlear implants can "improve" the life of a child born with an inability or severely reduced capacity to hear sound.

Given the results I have been exposed to, admittedly indirectly, there is no reason to believe that cochlear implants are close to being a real cure for the majority of people with hearing loss. Deaf individuals I have talked to mostly downplay or reject the value of even the more modern devices. Everything I have read sends similarly mostly negative signals. It remains an excellent option for some, primarily individuals already accustomed to hearing.

Diversity issues aside, it seems medically questionable to force the procedure on any individual. Considering diversity, the notion that individuals should be made or expected to conform is deplorable. This polar view certainly does not take into consideration the full range of complexities in the decision making process any parent must undertake in selecting medical treatment for their child. For now it is all I can hope for that parents consider more than a shallow sense of "normalcy" as an appropriate deciding factor.

One place where adjustments to language rather than the individual seems to be more readily accepted is in application to children with challenges such as Autism (Visual Language in Autism, Shane & Weiss-Kapp). Gestures and images work well for most, text works for a few. Because Autism falls along a spectrum with many individual factors involved there isn't a single "right" approach, only approaches that work, or work better.

The expressiveness of such languages is functionally limited. Symbol order follows formulas with exceedingly simple syntax. One concern with such watered down language is that it might "blunt" the development of speech. Clinical experience applying these adaptive languages indicates the opposite, that they help bridge difficult conceptual gaps resulting in better acclimation to other peripheral language skills rather than being a "crutch". The approach proposed in Visual Language in Autism embraces three ways to deal with barriers, each when it is most effective: Fit-it, Compensate, Bypass.

The concept of shibboleth, the use of language to distinguish between insiders and outsiders or imposters, like written language, predates the birth of Christ. It is a form of discrimination based on language, a way to judge. While shibboleth has legitimate defensive purposes, like any other technology is has a wealth of abuses. Language is naturally barriered: hearing, vision, light, concentration, a noise free environment, and literacy are all examples of things without which one may run into communication challenges.

Certain combinations of technology lead to erosion of these barriers. Candles allowed early man to read in the dark, focal lenses allow some with visual challenges to see what would otherwise be blurry. Electronic text is perhaps the closest we have come to a communication panacea to date. Electronic text can be automatically translated between languages with varying levels of reliability that is improving each year. It also can be read out loud, bridging the visual barrier all together. Electronic text can be printed to physical pages and braille. It can stream across the screen helping to compensate for hearing barriers.

It is in this one sense that perhaps luckily, alphabetical text seems to have been the right long term choice. Optical character recognition for converting printed text into electronic is by no means trivial, but it at least seems much simpler than the process for programming a computer to parse a more visually rich and directionally diverse written language.

None the less, romantic ideas of visual languages are fun to explore for some. I have long been a fan of The Elephant's Memory and other similarly visually engaging approaches. Sense of shape and direction are very stagnant in our modern language. Even ideogram languages, such as Japanese and Chinese have fairly stagnated sense of visual space. Our words have become reliant on images to make up for the things they cannot express as elegantly, though infinitely more efficiently. This brings a third "E" criteria to my standard list: Effective, Efficient, and Elegant. Each of the three are interrelated, but distinct.