Saturday December 01, 2007 | Automated Software Engineering Research Group @NCSU |
|
All
|
General
|
Research Skills
|
Technical Writing
|
Submission Madness
|
Idea Generation
|
Research Evaluation
|
Research Ethics
|
Research Community
More on formal writing before one-on-one meetings
Here is the definition on formal writing: (1). The formal writing includes the text that you can turn into
a part of your future paper submission directly or with minor polishing. If you
just write in some high-level bulleted points like those in slides, this type
of writing is not formal and not acceptable in terms of formal writing. (2). Because our group uses LaTeX as the format of writing
papers, your formal writing needs to be in the LaTeX format. If you don?t know
how to use LaTeX in writing papers, take a look at http://people.engr.ncsu.edu/txie/publications/writingtools.html Especially on which software packages to use for editing and
compiling LaTeX source files. (3). Because our group uses CVS to keep track of revisions and
allow collaborative writing, your formal writing needs to be put in our
research server?s CVS repository. Basically after you set up CVS, you can create
a subdirectory under /cvs/root/papers/ with the naming convention of
?lastname-conferenceorworkshopname? (e.g., acharya-FSE07). If there is no
specific conference or workshop to aim at currently, you can put the name of
your project/tool/topic in the place of ?conferencworkshopname?. For info on
how to set up CVS and use Eclipse to checkout CVS, take a look at: http://ase.csc.ncsu.edu/server.html#cvssetup Then your submission of your formal writing is an email
including some words like ?My formal writing so far is included in the CVS
directory XXXXXX. You can check it out.? Basically you can view the formal writing that you submit before
our one-on-one meetings as a portion of the paper that you are going to submit
eventually. Week after week, you will expand the draft by filling in additional
text that describes what you have done in the preceding week(s) and in the
upcoming week(s). Note that initially or early in the phase of your formal
writing, you shall write the abstract, introduction, example sections early on.
In addition, you may also start writing the related work section when you read
other researchers? papers early on. Writing these preceding sections
doesn?t require any tool implementation or experiment. Then along the way of
week-by-week work, you fill in the approach/implementation sections when you
have more implementation details figured out and more development work done,
you fill in the experiment setup and design sections when you try to set up
your experiment, and you fill in the experimental results section when you
finish producing experimental results, ? This mechanism is to fix several issues being faced nowadays. (a). students tend not to write serious/formal text along the
way but put a lot of efforts in formal writing immediately before a submission
deadline. Then the students cannot get helps from me on their writing early on.
(b). students tend not to disclose sufficient technical details
or progresses of their projects along the way during one-on-one meetings week
by week. Often immediately before the deadline, some students gave me
?surprises?, disclosing to me that they didn?t do some part that they were
expected (by me) to do or they did something in an un-optimized or incorrect way;
then it is often too late to fix these issues when getting too close to a
deadline. (c). when students don?t write things down in formal writing,
they don?t have good feeling in the approach/tool design, experiment design, ?
I often come up with good new ideas when I formally write down ideas in my
proposals and I expect students to enjoy similar benefits by doing formal
writing along the way.
Written materials prepared before one-on-one weekly meetings
I recently talked to a colleague, Dr. Nagiza Samatova, who kindly shared her experience in training students' writing, and inspired by her way of training, I have tried to install a similar mechanism in my research group. I suspect that it will solve some students' issues in delaying writing in the last minute and turning their research as a black/grey box to me. Below is adapted from my email sent to some students in my group who have already had some concrete research projects ongoing:
Before one-on-one student meetings, the advisor requires the student
to bring formal technical writing on the things to be discussed: the written
materials later will be turned into a part of a paper submission so it is not
wasteful or just specific for being used in one-on-one meetings. For example, -- if you plan to discuss a new idea that you may have, write paragraphs describing it, which can be turned into the introduction section, example section, or approach overview section of your future paper. -- if you plan to discuss about
design and implementation of your approach, write paragraphs describing these
designs or implementations, which can be turned into the approach and
implementation sections of your future paper. -- If you plan to discuss other related papers that you read, write paragraphs describing them and the differences of them with your own approach, which can be turned into the related work section of your future paper. In any case, you shall prepare
your writing and present it to me along the way of weekly one-on-one meetings
rather than a big bang in the end immediately before the deadline. Doing so can
allow me to (1) give you early feedback on your work and writing and to (2)
keep track of your work since currently your work?s technical progress is more
a black/grey box to me. In addition, this mechanism
would be also very helpful to yourself in keeping yourself in having the habit
of writing things down more formally (when you try to write things down more
formally, you can have a better idea and generate new good ideas). I expect you to send me an email
telling me the sections/paragraphs in LaTeX in your paper in a specific CVS
paper directory ** not later than the same morning ** of an afternoon
one-on-one meeting. I don?t accept informal writing being put in the body of an
email message or any way other than the preceding specified way. If you cannot prepare such
writing before a one-on-one meeting, I would suggest you to cancel or postpone
that week?s one-on-one meeting with me. If you cancel or postpone too many
weeks? meetings, the implication of reflecting your work progress/performance
is self-evident. Posted by txie ( Oct 13 2007, 02:36:54 PM EDT ) Permalink Comments [0]
Can we learn from Dr. "House" in doing research?
I enjoy watching Fox's House TV series.
I find the problem sovling skills and creative thinking there to be inspiring for us in doing research.:) I enjoy watching how Dr. House advises his "students". I hope to learn the good poritions of his advising styles.:) " DR. GREGORY HOUSE (Hugh Laurie) is devoid of bedside manner and wouldn?t even talk to his patients if he could get away with it. Dealing with his own constant physical pain, he uses a cane that seems to punctuate his acerbic, brutally honest demeanor. While his behavior can border on antisocial, House is a brilliant diagnostician whose unconventional thinking and flawless instincts afford him a great deal of respect. An infectious disease specialist, he thrives on the challenge of solving medical puzzles in order to save lives. For the past three seasons, House has shepherded an elite team of young experts who helped him unravel diagnostic mysteries. In addition, he has a good friend and confidant in oncology specialist DR. JAMES WILSON (Robert Sean Leonard). There?s some volatile chemistry between House and DR. LISA CUDDY (Lisa Edelstein), the Dean of Medicine and hospital administrator; the two are in constant conflict over House?s duties and unconventional behavior, but even she would admit that his brilliance is worth the trouble. In the Season Three finale, the set-in-his-ways House was confronted with a series of major changes to his team. Neurologist DR. ERIC FOREMAN (Omar Epps) left Princeton Plainsboro because he didn?t want to turn into House; House randomly fired old-money intensivist DR. ROBERT CHASE (Jesse Spencer), claiming he learned everything he?s going to learn in the past three years, or nothing at all; and immunologist DR. ALLISON CAMERON (Jennifer Morrison) resigned, knowing House will be completely unaffected by her decision. As Season Four opens, House is without a team to contribute to the perplexing medical cases he undertakes, and Cuddy and Wilson are adamant that he recruit new fellowship candidates. After 40 applicants applied for the newly vacated spots on his team, a group of five doctors -- played by Olivia Wilde, Kal Penn, Peter Jacobson, Anne Dudek and Edi Gathegi -- have emerged as finalists vying for the coveted and hotly contested openings." Posted by txie ( Oct 13 2007, 02:17:04 PM EDT ) Permalink Comments [0]
Reading papers - 5 line summaries!
Dr. Xie maintains a very nice bibliography on Mining Software Engineering. We read lot of papers, but with time, tend to forget them. How about having a 5 line summary for each of the paper we read as a part of literature survey? I actually maintain a document which does exactly this and find it very useful. So next time I forget whats in a paper, I go to my document and look for the 5 line summary, and I immediately know what the paper talks about. I dont need to read the paper again. Another useful side-effect of this exercise is when you write related work for any of your papers or thesis. In conferences, when you talk to other researchers, they usually ask - "Have you seen paper X? How is your work different from paper Y?" and its bad not to know some really relevant related work!
Most well written papers, can be read in about 15-30 mins and summarized in about 5 lines. In my field, most papers have a motivating example after introduction. For a well written paper, a reader should get the idea of the whole paper when he completes reading the Example section! So the way I read a paper is - read abstract, look at the conclusion, then read introduction (very fast), and then the example section. This process takes about 15 mins. Then I skim through the framework, implementation, and evaluation details. I spend further time on the paper, only on need basis. Then I summarize the whole paper in about 5 lines! During early years of PhD, it might be beneficial to read the whole paper to learn the art of writing papers... but after getting a hang of writing papers, quick paper reading will be a useful skill! Comments? Posted by mpachary ( Oct 11 2007, 08:47:38 AM EDT ) Permalink Comments [1]
Promoting Research Group Spirit and Peer Student Support
Earlier I didn't emphasize much on
research group spirit. Recently I realized its importance and tried some
measures to promote research group spirit. 1. Allow students to volunteer to
take on some services in the group. In the past, I (as the advisor)
took on most of the services in the group including maintaining the
group web pages, coordinating the group meetings, etc. Then students might feel
like being managed without feeling to own the research group. In addition, I am
too busy in doing these types of things and the students don't learn how to
organize things or manage things: an important skill in their future career. *. Group Webmaster (news, group Web
page, pictures, etc) I found this mechanism works pretty
well. For example, recently when a visitor from industry gave a guest lecture in
my course when I was out of town, I asked the industry/visitor coordinator to
organize student meetings with the visitor by introducing our research and
doing demo; the whole process was organized by the coordinator with help from
other students. The process went well and the students can also improve their
independent skills: when the advisor is not around (in the future after they
graduate, their advisor won't be around!), they can still successfully carry
out things. But I still need to figure out a way
to encourage students to send emails in our group mailing list, whose emails
are primarily sent by myself. 2. Acknowledge and honor those
students who made great achievements in research so that these students can
feel being recognized and other students can learn from these students and try
to catch up. Jiawei Han's group honors the best-performing students each
semester after students submit their research performance summary
for the semester. Recently our research group also held voting among students
(each one vote) and myself (with two votes, as suggested by one student, saying
that my judgment would be more comprehensive). In the end, we voted one golden award winner and two
silver award winner (with the same number of votes). 3. Besides borrowing Jiawei Han's measures, I also tried to promote peer support among the students in the group.
Earlier the whole group activities centered around me, including reviewing their
paper drafts, giving feedback on their research, etc. I would hope to set up a
peer support system so that students can help each other and learn from doing
so. Since some time ago I encouraged students to do proof reading each other's
papers, and help each other. I will think of more other measures in promoting
peer support. 4. As a routine practice in many
research groups, asking students to present their own work or other related
work by other researchers is quite valuable. Earlier I used the group meeting
time slots to go round-table debriefing and I found it not that worthwhile in
spending time. Nowadays, instead, in each group meeting, each student makes a
presentation and then other students and I give feedback either on the content
or presentation skills. Again, in this way, the group meetings shift from being
dominated or driven by myself to being managed by students themselves. I will think of more other measures
in promoting peer support and group spirit. If you have any comments, you are
welcome to discuss here.
A mechanism to help students in indepenent thinking
As a graduate student, you are supposed to grow to be independent
along the way. To help you to do that, I ask you to allocate the last 10 mins
of the one-on-one 30 mins meeting time slot to train your independent thinking.
Basically in these 10 mins, you should tell me your thoughts on answering one or more of the following questions: --- ?What to do in more details for the current project idea if it is not that detailed or clear enough?? --- ?What is the next good idea (beyond the current one) that you should work on?? --- ?What would you do in the next big phase (either in 1, 2, 3, 4, or 5 years)? ? What does this mechanism mean? You should think about future research ideas ALL the time!! I have been always thinking about new research ideas all the time. You should do the same rather than relying on me to tell you what to do next. In addition, you should actively read more and think more with the explicit goal of generating ideas for your future research. It is not acceptable that you tell me that you haven?t thought about IT when we reach this 10 min slot, because you are supposed to think along the way (jogging, walking, taking bus, taking shower, sometimes driving but be careful, ?)
Levels of research committments
Four levels of research commitments can be classified for students (see below). Each of you shall classify yourself to one of these levels. Each of you shall try you best to reach or keep in Level 1. Not many students are currently
reaching or staying in Level 1. There are positive iterations when you reach Level 1. When a student is efficient and effective in finishing research tasks, the advisor will work with the student on coming out new good ideas for the next research project. If the student stays on an existing old research project for a long time, before the advisor works with the student on a new good research project, the advisor will wait for the student to finish the old one up or wait for the student to tell the advisor that the existing old project has no hope and the student would give it up. Some students fall into Level 4. It is a very dangerous situation. Staying on Level 4 can cause you to stay in the program for a long time without producing any research results. Level 1. Self-propose timeline in research tasks and often succeed in accomplishing the research tasks Level 2. Self-propose timeline in research tasks but often lag or fail in accomplishing the research tasks Level 3. No self-proposed timeline in research tasks but be willing to discuss with the advisor in research timeline Level 4. No self-proposed timeline in research tasks and even no responses upon the advisor?s requests on checking research status Posted by txie ( Aug 03 2007, 12:27:58 AM EDT ) Permalink Comments [0]
Advice on making submission deadlines
I found that some students who are supposed to drive their research and
preparation of a certain submission draft are not active or responsive enough. Below is my advice on dealing with the issue.
1. Students need to early on take full advantage of the advisor and other colleagues (in the co-author list) in helping improve the draft and the work. As I told students in our group, students should write the whole draft (when there are peer colleagues/students in the co-author list of your paper, you may coordinate with them to ask them to write some sections of the paper). Enforcing students to write all sections can help train their capability of independently writing the whole draft. Of course, the advisor will help you by giving you suggestions on how to revise your draft. That doesn't mean that you need to submit your draft for your advisor to review only after you finish the whole draft. It is great if you can finish your draft very early on and send your whole draft to your advisor. But more commonly many students feel tight in making their drafts ready. Then the students need to make efforts to gather early feedback from the advisor by giving section by section to the advisor for review comments and feedback if they cannot prepare their full draft early on. Like in bug finding, the earlier that a bug is detected, the better off you will be in fixing the bug. In all, try to get early feedback from the advisor or co-authors incrementally with available sections early on rather than putting off sending your writing to them very near the deadline. In the latter case, the advisor may not have enough time and you may not be able to incorporate the feedback to improve the draft. 2. Students need to be **responsive** to the advisor or colleagues. Responding your advisor's emails should be on the top priority if your advisor's emails explicitly asked for responses with questions. As you can see, I always give rapid response to students and colleagues. As I discussed in the group meeting, in some other groups either inside or outside NCSU, students may complain that their advisor is slow in responding their emails. In our group, the other way around happens often, believe it or not! If you are too busy and cannot spend time on some task mentioned in the advisor's email, you can simply respond so and then the advisor or the colleagues can know it and make alternative arrangements or schedule their time line before the deadline. The advisor's goal is to help you to make these deadlines, produce good work, grow to be independent enough, and then graduate, find your desirable job. Being not responsive or not effective in making deadlines or making progress in your work can hurt yourself much more than anyone else. That is why I told you "Do you want to make the deadline. If not, it is totally fine to me." That is, it is **you** who want to make the deadlines and it is **you** who need to drive your research, not anyone else. Good luck on your deadline catching! Posted by txie ( Jun 09 2007, 10:50:25 PM EDT ) Permalink Comments [0]
SE conference map came alive
Check out the Upcoming Software Engineering Conference Map created and maintained by Sung Kim and Tao Xie!
It came alive several days ago! Posted by txie ( Jun 04 2007, 02:20:01 AM EDT ) Permalink Comments [0]
Papers accepted at ESEC/FSE 07 and ICSM 07
Mithun has a paper accepted at ESEC/FSE 2007 (17%), a top 1 conference in software engineering (together with ICSE).
My Peking University colleagues have a paper accepted at : ICSM 2007 (21%), a highly reputed conference in software maintenance. Congratulations on their achievements! Posted by txie ( Jun 03 2007, 02:39:54 PM EDT ) Permalink Comments [0]
One pitfall in writing the related work section
In the related work section or elsewhere in a paper, many people tend to
compare the proposed approach with the existing approaches in a shallow way,
making the justification weak.
For example, your approach may be a static analysis approach and when you compare with previous dynamic analysis approaches, it is bad for you just say "Our approach based on static analysis are different from the existing approaches because they are based on dynamic analysis." It is not enough. You should describe what observable inputs/outputs (benefits) that the internal technical differences in your approach could bring. Here, it is better to say "Our approach based on static analysis are different from the existing dynamic analysis approaches because they require runtime setup and good sets of test suites, which are not required by our approach." You can see that the difference is on the observable inputs. When you compare your dynamic analysis approach with existing static analysis approaches. You can focus on the difference of the observable outputs. You may say "... because our approach uses precise runtime information, reducing false positives that are produced by existing static analysis approaches." This issue has been raised in reviews for one of our past submissions. "you omit effectiveness rates of a lot of tools in favor of "ours technically works differently on the inside". That is not quite good enough; the comparison should be on result, not just on "we are technically different"." You should go through your related work section to see whether you described the differences in observable results/benefits outside beyond just describing technical differences inside. Posted by txie ( Jun 03 2007, 02:32:38 PM EDT ) Permalink Comments [0]
The year-end picnic (Pig Pickin')
Dr. Tao Xie, Mithun, Yoonki, and JeeHyun joined the 'Pig Pickin' picnic as of 04-27-2007.
We had great food and enjoyed meeting with other students. The followings are the picture of us !! pic 1 pic 2 Posted by jhwang4 ( Apr 30 2007, 11:33:24 AM EDT ) Permalink Comments [0]
Answers to Global Educator Magazine, Student Section
The Global Educator magzine addresses students of science and engineering. The
focus of the magazine is to provide information to students about educational
institutions (specifically focusing on engineering and technology) in India and
overseas, inform them about developing opportunities in
engineering, offer practical tips on how to manage education abroad, carry
interviews and profiles of achievers in the field of science, technology,
engineering and business as role models.
Below are the answers to the interview questions posed by the Global Educator to me. They will be published in a future issue of the magzine. 1. Could
you tell us about your career so far? I am currently
a PhD candidate in the Department of Computer Science at North Carolina State
University, Raleigh, North Carolina. I completed my Master of Science degree
from the same university in 2003, and Bachelor of Engineering degree from
National Institute of Technology - Karnataka (previously KREC), Surathkal,
India, in 2001, all in Computer Science. During my graduate studies, I have
interned at NEC Europe Network Laboratories, Heidelberg, Germany and IBM T J
Watson Research Center, New York. In 2007 summer, I will intern at Microsoft Center
for Software Excellence, Seattle. 2. How
did Software Engineering interest you? I like
exploring research ideas that have immediate practical applications;
innovations that create impact and are adopted by a very large community. Specifically,
my interests in Software Engineering lie in the area of Automated Software
Testing and Verification. It is indeed a very satisfying experience to come up
with research methodologies that ultimately improve the developer productivity
in the software industry. I was introduced to this fascinating world of
Software Engineering through the Automated Software Engineering research group
(http://ase.csc.ncsu.edu/) led by my advisor Dr. Tao Xie, of which I am
currently a part of. In simple
words, my doctoral research focuses on developing efficient techniques to
automatically find bugs in software programs. Bugs can have serious
implications ? illegal access to a personal computer, failure of a life-saving medical
device managed by software, complete network shut down of a large organization,
to name a few. Bugs are mainly caused when programmers fail to adhere to
certain application-specific, non-syntactic rules in the code that they write.
These rules, when documented and available, can be checked against a given
program using sophisticated software verification tools such as model-checkers.
So far, my doctoral thesis focuses on addressing two problems. (a) Writing
rules formally for software verification is hard and error-prone (b) Application-specific
rules may not be documented, hence being unavailable. I have proposed and
implemented a framework that makes specifying rules for software verification
easy. Furthermore, I apply data mining techniques to automatically extract such
rules directly from program source code or program execution runs. These rules
can then be used to find bugs. 5. How
has your experience been studying and conducting research in US university? http://www.csc.ncsu.edu/faculty/xie/advice/ Posted by mpachary ( Apr 27 2007, 09:10:29 AM EDT ) Permalink Comments [1]
How to meet your advisor?
In my view, the major purposes of individual (i.e., one-on-one) meetings include several parts:
(a). Keep the advisor informed on some research details (either the work being done the period before the meeting or the work to be done in the next time period) so that the advisor can give advice or guidance on them (b). Brainstorm new research ideas and get feedback from the advisor (c). Allow the advisor to know that you are making progress and going on the right direction/track. (d). Any other things that the student wants to talk to the advisor (like general career advice, things that happened recently in the group, department, and the research field) The first advice for students on meeting the advisor is "do sufficient preparation before your meeting with your advisor". I can tell that some students sometimes didn't prepare for their meeting with me. They came in my office and didn't have a meeting agenda in mind. After they used one or two sentences very briefly to inform me what they have done and they are going to do, they stopped and had no specific things to talk about during the rest of the meeting. Students should list about things to be discussed in mind or in paper. In my early day's meeting with my advisor, I brought in printed hardcopy slides or some note printout. In the beginning of the meeting material, I put a list of items to be discussed if there are more than one item. I have thought carefully how to make full usage of the 30 mins (later 45 mins or 60 mins) that my advisor allocated for me. I usually spent some non-trivial time in preparing materials for my meeting with my advisor (this preparation could also help me to organize the work being done in the passing week and plan out what I am going to do in the next week). There are two general types of students in terms of their style of meeting the advisor: 1. Have not much to say during the meeting with the advisor 2. Have too much to say during the meeting with the advisor The first type is more common, I think. Many students often cannot use up the time slot allocated for them (either 1 hour, 30 mins, or 15 mins). They don't seem to have much to say with the advisor. I think a wise student should take full advantage of the allocated time slots. For both types, the high-level quesiton that the students should ask themselves is: "What do you want from your advisor during this meeting?" Your advisor also had the same question "what do you want from me?". This question would be especially helpful for the second type of students. Usually the second type of students wants to expose to the advisor every tiny detail of the work that they have done or every tiny detail that they are going to do. If a student usually finds that he or she cannot finish the meeting within the time slot, there are two options: one is to request a longer time slot if the student finds that he or she has indeed important things to talk to the advisor, and the other is to rethink whether some of the stuffs that the student is telling are important for the advisor to give advice. That is, after the student explains the tiny details, would he or she expect the advisor to give specific useful advice? Or would the student expect certain advice on the tiny details? If not, than probably the student doesn't need to explain that levels of details to the advisor. One downside faced by the second type of students is that when a student overwelms the meeting with so many tiny details, the big picture is lost and the advisor may lose the opportunity and time to talk to the student on high level more important things. The first type of students is way more common. They tend to keep many research details to themselve, not discussing these details with the advisor; this way may be acceptable or even preferable when the students are in a late stage that they can have quite high confidence of doing **right** things in **right** ways independently. Then they can focus on high level big pictures and other high level things during their meetings with their advisor. In my late stages of my Ph.D. program, I pretty much go for this way. But for students in early stages, they should discuss more research details with the advisor. One way to fix this issue of not much to say is to do more preparation before the meeting, as mentioned earlier! Before the meeting, think about what you want to disucss to use up the allocated time slot! Another possible reason that some students have not much to say is that they are busy with their course work and other legitimate things, not being able to spend much time on developing the reseach project or making enough progress; this situation occurs very often when students are in their early stage, taking non-trivial course load. The advisor would often show understanding on the situation. But that doesn't mean that students would skip the meeting or have not much to say in the meeting. In my early stages of Ph.D. program, I didn't or couldn't spend a lot of time in developing research projects, partly because of taking courses and partly because of lack of good research ideas to work on (I struggled on coming up good reseach ideas for quite a while in my early stage of my Ph.D. program). But I still kept routine meetings with my advisor, talking about ideas that I have, talking about my understanding of some papers, talking about other things ... Of course, I needed to spend time on preparing for such meetings especially when the meeting didn't focus on speciifc research details that I developed. In my Ph.D. program, when I had a meeting with my advisor, I wouldn't leave till I used up all the time allocated to my meeting. When I finished technical parts, if some time still remains, I would talk to my advisor on non-technical, causual stuffs like research community, high level career advice, the advisor's view on certain things, ... In summary, I hope students can use up and take full advantage of their meeting time slots with the advisor. Posted by txie ( Apr 26 2007, 11:42:09 PM EDT ) Permalink Comments [2] |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Some pictures of ASE group as follows!!
pic 1
pic 2
pic 3
Posted by jhwang4 ( Dec 01 2007, 02:07:13 AM EST ) Permalink Comments [0]