Automated Software Engineering Research Group @NCSU : weblog

Thursday Apr 19, 2007

Research skills required to conduct research (on automated software engineering)

In the past, I told my students that three levels of research performance/skills could be assessed (an analogy with CMMI levels?:):

*. Level 1: Independently carry out/implement research ideas (including implementing tools and doing experiments) given by the advisor or other more senior people

*. Level 2: Independently write research papers

*. Level 3: Independently generate (good) research ideas; sometimes, coming up new research ideas can be categorized as (1) coming up with a new research problem or (2) coming up with a new research solution to an existing research problem; the former is often more challenging

"High effectiveness" and "high effciency" are two cross-cutting quality attributes for these three levels.

When a student starts working in our research group, I would expect the student has reached Level 1, which means the student shall have strong programming and debugging skills. My major role is try to train/help a student to reach Levels 2 and 3. Level 2 is not so difficult based on my advising experience (but I admit that teaching students on how to write well and putting so many red marks on their writing and going through these red marks with the students are a very time-consuming and yet rewarding process). Unlike many other professors, I ask students to write the whole paper draft even for their first paper because I believe that if students don't get a chance to write much and make mistakes (later corrected by me), they can never learn how to write well independently. I will talk more about improving technical writing in some future entries.

Reading other people's papers, playing around existing tools, and getting hands-on experiences are important factors for helping to come up with good new ideas. But that is not enough. On another future entry, I will discuss the issue of educating students to think and come up new ideas. This task is a very challenging one and I am still struggling on finding effective solutions.

Posted at 07:20PM Apr 19, 2007 by XIE, TAO in Research Skills | Comments[4]

Comments:

I think find out a new problem and find out a new solution are not at the same level. But I also can't tell which one is more difficult. Maybe people without much knowledge on others' work can generate good idea :) Now I am prefer define a problem by myself and find out solution latter :)

Posted by Nilson on April 20, 2007 at 05:03 AM EDT #

sorry, Now I am prefer define a problem by myself and find out related work latter :)

Posted by 221.218.17.185 on April 20, 2007 at 05:04 AM EDT #

I still think finding a new (good) problem is more difficult. In some cases, "people without much knowledge on others' work can generate good ideas". But I think people with much knowledge on others' work would have a higher chance of generating good ideas, because many good ideas are applications or migrations of existing ideas being applied on a different problem or area; without knowing existing ideas, people cannot generate this type of related new ideas.

Posted by txie on April 20, 2007 at 12:40 PM EDT #

It's wonderful to see this post and this set of comments. My experience shows that finding good problems in software engineering research is usually challenging, time-consuming, fun, and complicated. It requires a mix of knowledge, experience, and creativity -- and these can mix in at different times in different ways. Just as people have varied learning styles, people have varied ways of defining good research problems. The process of finding good software engineering research problems is rarely linear -- it takes some bouncing around at many levels, it takes extreme diligence, and (I'm sad to say) it takes a little bit of luck sometimes.

Posted by David Notkin on April 23, 2007 at 04:54 AM EDT #