Creative Artificially-Intelligent Agents for the Arts: An Interdisciplinary Science-and-Arts Approach
Creative Artificially-Intelligent Agents for the Arts: An Interdisciplinary Science-and-Arts Approach

coordinator: Jonas Braasch
Advisors:
Selmer Bringsjord, USA
Ted Krueger, USA
Johannes Goebel, USA
Pauline Oliveros, USA

Artificial Intelligence (AI) has made impressive progress since its start in 1956. It now influences our
daily lives, as AI systems are an integral part of consumer technology today, from SIRI to automobiles
to Semantic Web. However, while AI systems can be very successful if they are precisely told what
to do (e.g., perform a parallel parking task, play chess), they are usually useless if the objectives are
not clearly spelled out. They can learn along a precisely given trajectory (e.g., to learn to understand
spoken text or compose an instrumental music piece in the tradition of JS Bach), but they don't break
rules to produce something more exciting. Deep Blue can play chess, but if you present it with a
game implemented on a chess board, it will be lost. In short, machines are simply not very creative.
The idea of this white paper is to form an intellectual think tank to overcome existing roadblocks and
investigate alternative strategies in AI. Among the items that will be discussed is the implementation
of design oriented processes for AI systems. Artists and designers often work on a less hypothesisor
goal”driven approach as compared to scientists and engineers; they pursue an open”ended, purely
experimental approach instead, where the outcome of each phase informs the next one, not
necessarily having a fixed goal in mind. Along with this approach, there is a need for better AI
evaluation systems that can judge the outcome more freely than just examining the results along an
externally given set of rules. Using the experience of artists with the abstract, can we make agents
more creative by allowing them to be continuously evaluate what they accomplish? How can we
create AI systems that can develop and evaluate their own concepts?
Part of this discussion will include the creation of a network for more complex AI systems that
simulate several areas of the brain, or the abstract AI equivalent, simultaneously, by using a
meta”concept to connect existing AI modules using a UDP protocol in a computer”cluster network.
Another central aspect are systems that can draw on different algorithms to perform a task, making
the selection part of the creative process. Along the same lines, we can look into web data”mining
methods that allow these machines to receive information beyond what is given to them by the
experimenter.

Action Items

1) Complex Systems with Modular Architecture and Interchangeable Data Format

Roadblock: A lot of specialized software exists to simulate certain aspects of intelligence from computational auditory scene analysis algorithms to logic prover. In general, it is still very difficult to combine these specialized systems to complex systems simulating multiple parts of the central nervous system.

Opportunity: Enable a dialog to find better ways to standardize communication protocols between different systems and to port algorithms to a unified platform for creative intelligent systems

Proposed Action: conference or symposium to start dialogue

Stakeholders: university-based groups, gaming and entertainment industry

2) Agents that can handle abstract media and techniques

Roadblock: In engineering and science related disciplines a common approach is to copy the human body in both form and functionality. Honda's Asimo robot and Kaist's Hubo are good examples for this approach. Sometimes abstract solutions provide a better functionality, for example robots from children and science fiction movies are often more sociable, but artists and designers often lack the technical expertise of engineers

Opportunity: bring both groups together to build on each others' strength to build highly functional, powerful but abstract systems.

Proposed Action: conference or symposium to start dialogue

Stakeholders: university-based groups, gaming and entertainment industry

3) Need of creative synthetic characters that can develop new concepts

Roadblock: Over the last 40 years we have develop artificially intelligent agents that can produce creative work within a given context (e.g., compose music in the style of J.S. Bach), but system that go beyond this and develop their own concepts (e.g., a new composition style) do not exist yet (at least not in the sense that they can reflect and justify their actions).

Opportunity: bring together transdisciplinary groups of artists, psychologists, and engineers to elicit how humans complete these tasks and find ways to implement this knowledge to artificially intelligent systems.

Proposed Action: conference or symposium to start dialogue

Stakeholders: university-based groups, gaming and entertainment industry

USING 'PROCESSING' AS A STIMULUS FOR PRODUCING STEAM
USING 'PROCESSING' AS A STIMULUS FOR PRODUCING STEAM

Ron Brown
http://ron-brown.artistwebsites.com

Computational Artist/Educator/Programmer
November 8, 2012

The Challenge and the Opportunity

Sir Ken Robinson [1][2] has called for a paradigm shift in our educational system away from the use of standardized testing and behavior modification drugs on our youth to one of enquiry and creativity in the arts and sciences. I think a great opportunity exists today to achieve many of the goals he advocates utilizing tools from the open-source community, in particular, a computer programming language called 'Processing' [3]. I believe Processing can be used as a stimulus for merging the worlds of art, math, science and technology to meet the challenge of changing paradigms.

Processing

Processing is an open-source (FREE!) programming language developed at MIT by two graduate students (Ben Fry and Casey Reas) that is targeted for visual artists who would like to utilize digital media in their endeavors but who lack computer programming skills. It has become so popular that several circuit board manufactures have developed boards that can use Processing to obtain sensory data and/or to control motors (think 'robots') and other devices. In addition, Processing can be used to obtain data from the Kinect 3D camera (Xbox) for visual explorers to investigate the realm of 3D interactive media.

Course

During the spring of 2012 I had the opportunity to teach a 'Programming with Processing' course at a small independent Buddhist high school [4] in Ottsville, PA. The course met twice a week for twenty weeks where each session was one hour twenty minutes long. The students varied widely in academic skills and backgrounds and came from several different school districts but all stated they were glad they took the course.  One student stated he had known nothing about programming prior to taking the course but is now considering computer programming as a career. The only prerequisite for taking the course was the desire to learn a computer language ”" no other strings were attached. On June 22, 2012 I was one of several presenters at a STEM to STEAM conference held in Baltimore, MD, where I talked about the course and showed images created by the students [5]. Some of my own artistic explorations using Processing can be found at [6].

Potential Participants

I think a course using Processing to create images would be greatly appreciated by 'gifted' students, by students who are comtemplating dropping out of school because they are bored, students who may have gotten in trouble with the law, other 'high-risk' students and students in the 'general' population. The growing 'home school' movement should not be exempt from the opportunity of utilizing Processing in its curriculum either. Another targeted population that should not be excluded is that of teachers who would like to explore Processing themselves and learn how it can be used in their own classrooms to produce some STEAM. I feel strongly that the only prerequisite for any student should be the desire to learn a computer language. Let the student have the opportunity to fail in a 'safe' environment and learn from his/her mistakes. Computer programming is an unforgiving endeavor and attention-to-detail is a must that is soon learned. But, it's FUN!

On a more grandiose scale, curricula based on Processing could be developed for interested school systems starting at the sixth grade level or below and extending through the senior high school level. Estonia (birthplace of Skype) has implemented a plan to introduce computer programming to students beginning at age 6 [7].

Suggested Actions

Suggested Action #1: Finding 'Qualified' Programming Instructors

Barrier: Too few secondary teachers have any substantial computer programming experience ”" no matter what subject(s) they are 'certified' to teach. Unfortunately, there are many experienced programmers who know different computer languages but they are not allowed to teach because they do not have a teaching 'certificate' to do so as required by the educational establishment.

Stake Holders: School administrators, federal and/or state legislators, Departments of Education, teachers who want to make their classes more exciting, teachers' unions, unemployed/retired computer programmers

Suggested Actions: During the last decade, many software projects were outsourced to developing countries, leaving many American computer programmers jobless. A large number of these former computer programmers could be recruited and learn Processing in a relatively short period of time ”" compared with someone with no previous programming experience ”" perhaps within one month or less. With appropriate incentives, these 'new' senior recruits could teach either younger teachers or teachers-to-be enrolled in colleges/universities/community colleges or teach within the secondary-school system itself. Educational instutions should be encouraged to offer computer programming courses in their STEM or STEM to STEAM programs.

Opportunities: Employing qualified senior citizens would be a win-win situation for those invited back to the workforce and for those who would benefit from their expertise. Current teachers will have the opportunity to learn skills they may have never experienced before.

Suggested Action #2: Eliminating standardized testing for STEM to STEAM endeavors.

Barrier: Standardized testing is so engrained in the educational psyche/system and is overly encouraged by testing services as well as textbook publishers and the Federal Government implementing laws such as 'No Child Left Behind'. Standardized testing does not measure/encourage 'creativity'. As Sir Ken Robinson has stated, 'creativity' is destroyed as a student progresses through the educational system.

Stake Holders: School administrators, federal and/or state legislators, Departments of Education, teachers' unions, textbook publishers, testing services, art galleries

Suggested Actions: The only prerequisite for taking any computer programming course should be the desire to learn computer programming. A student's 'grade' should not be based on any post-course test or battery of tests given throughout the course. Rather, a student's grade should be based on the student's completion of programs (called sketches in Processing) and/or one's effort in completing the assigned tasks (Example: one of my students had severe dyslexia but I have never seen anyone work/concentrate so hard to complete a sketch.). One goal of the course could be to show the students' art in an art gallery setting and use the proceeds of any sales to support STEM to STEAM projects.

Opportunities: Implementing a 'testless' course utililzing up-to-date technology would be an ideal case for one of Sir Ken Robinson's desires which is to eliminate standardized testing.

Suggested Action #3: Developing curricula spanning multiple grade levels.

Barrier: Knowledgeable curriculum developers may be few and far between. A related barrier is preventing the unintended consequence of institutionalizing computer programming as a mandatory subject. Just as Processing is 'open'-source, school administrators should be 'open'-minded when implementing such courses/programs. The only prerequisite should be the desire to learn a programming language.

Stake Holders: School administrators, teachers, curriculum developers, private grant foundations, Educational Departments

Suggested Actions: Instead of local districts being burdened with the task of developing their own curricula districts should utilize world resources. Agencies such as NASA and the NSF, along with private donor organizations, should take active roles in such development efforts.
Teaching Processing would allow those interested in robotics to learn the mechanics of motion/control using Arduino hardware. Those interested in the performing arts could expand their horizons by using Processing to interface with the Kinect camera for interactive performance utilizing music and/or other devices. To minimize costs, the use of textbooks should be minimized or possibly eliminated ”" internet resources are abundent and should be maximized to the fullest extent. Programs such as those being implemented in Estonia should be followed closely.

Opportunities: Networking with other like-minded individuals/organizations around the world would be very rewarding for all involved.

Suggested Action #4: Providing each student with access to a computer for programming.

Barrier: Many school systems do not have the monetary resources to provide individual access to a computer for programing purposes.

Stake Holders: School administrators, private grant foundations, parents, community groups, corporations

Suggested Actions: Many students have their own computers that have plenty of 'computer power' for programming purposes. Students who have their own computers should be encouraged to use them. Also, many companies upgrade their computer networks periodically and dispose of their computers en mass ”" companies should be encouraged to donate their used computers to their local school districts. Computers used for programming need not be connected to the internet permanently so the need for more net-working expertise would be minimal. Grants from private foundations should be sought.

Opportunities: Community involvement of parents, students, administrators, teachers and businesses would create a more caring community.

Suggested Action #5: Evaluating the proposed Processing course/curriculum.

Barrier: Many school systems do not have the monetary resources to monitor/evaluate the success/failure of new educational programs.

Stake Holders: School administrators, state Education Departments, program evaluators, graduate students, teachers' unions, parents, students

Suggested Actions: One or more graduate students in the near vacinity could be encouraged to write a thesis evaluting the proposed course/program and/or the State's Department of Education would have the task of evaluting the program.

Opportunities: Knowing the outcome of early courses/programs would lead to improved  courses/programs that would grow as technology changes.

Suggested Action #6: Providing resources to small independent schools and the home schooling movement.

Barrier: Small independent schools lack funds for such 'esoteric' courses/programs and would only be able to offer one if one of their limited staff had the skills/knowledge to teach the course. When I asked about teaching another 'Programming with Processing' course the following semester I was told that there was no funding available. I am quite sure that, had the school had adequate funds, I would have been able to offer the course again.

Stake Holders: Independent schools, parents, students, teachers, private grant foundations, corporations

Suggested Actions: Parents who send their children to private/independent schools, or, those that choose to home school, still pay federal and state taxes targeted toward educational programs ”" there is no reason their child should not have the same opportunity to share in STEM to STEAM endeavors as those attending public schools. Because of the current economic downturn it would be worthwhile for all school environments to share their expertise/resources with others.

Opportunities: Acts of kindness lead to acts of kindness. And, it's the right thing to do for our children and our future.

References

[1] Sir Ken Robinson's 12-minute summary: www.youtube.com/watch?v=zDZFcDGpL4U
[2] Sir Ken Robinson's 55-minute address: www.thersa.org/events/video/archive/sir-ken-robinson
[3] Processing website: http://processing.org
[4] Tinicum Art and Science High School: www.tinicumartandscience.org
[5] Innovate Our World Conference: www.innovateourworld.org/conference.htm
[6] My own artistic explorations using Processing: http://ron-brown.artistwebsites.com
[7] Estonia implements programming at age 6: http://gigaom.com/europe/estonias-plan-to-get-6-year-olds-coding-is-a-stroke-of-genius/

Processing Books

Algorithms for Visual Design Using the Processing Language by Kostas Terzidis (Wiley Publishing, 2009)
Making Things See ”" 3D Vision with Kinect, Processing, Arduino, and Makerbot by Greg Borenstein (O'Reilly, 2012)
Processing ”" A Programming Handbook for Visual Designers and Artists by Casey Reas and Ben Fry (The MIT Press, 2007)
Processing ”" Creative Coding and Computational Art by Ira Greenberg (friendsofED, an Apress Co., 2007)
Processing for Visual Artists ”" How to Create Expressive Images and Interactive Art by Andrew Glassner (A.K. Peters, Ltd., 2010)