•  
  •  
 

Abstract

SIMPLE: System Automatic Essay Assessment for Indonesian Language Subject Examination. Evaluation of study of a student is a very important aspect of an educational process. Evaluation is aimed at measuring the level of student understanding of the given lecture materials. Measuring student understanding of the course material, using essay-type exam, is generally used as the evaluation tool. In this essay-type exam, the student has to answer questions using sentences, whereby choices of possible answers are not indicated. The student has to answer the questions with his/her sentences. The answers may vary, since it reflects the student’s best thoughts of the materials. One of the weaknesses of essay-type exam is the difficulty to grade the answers and it tends to be time consuming. Currently, automatic grading systems that may speed up the grading process, are being developed in many research institutions. The method used to grade, varies form one system to another, and one of the popular system is the Latent Semantic Analysis (LSA). LSA is a method of grading essay by extracting words and representing the sentence in the form of mathematical or statistical formulation, from a text with a relatively large number of words. The grade of the essay is determined, by matching the important words to a group of words prepared by the human rater. This paper describes an effort to developed LSA, enhanced with word weighting, word order and the word synonym to improve the accuracy of grading. This system is called SIMPLE. SIMPLE is used to grade answers using bahasa Indonesia. The exam is carried out on-line through the Web. From the experiments conducted, for small classes, the conformity of grade compared to the grade of human rater lies between 69.80 % – 94.64 %, and for medium size classes the conformity lies between 77.18 % - 98.42 % with the human rater. These results are roughly proportional with the result of LSA system, which grade essay given in English.

Bahasa Abstract

Evaluasi hasil belajar menjadi komponen yang sangat penting dalam proses pembelajaran, karena hasil evaluasi merupakan indikator dari pemahaman siswa terhadap materi yang ajar yang diberikan. Ujian dengan jawaban dalam bentuk essay (esei), umumnya digunakan sebagai salah satu bentuk evaluasi untuk menentukan kemampuan siswa, dimana pilihan jawaban tidak diberikan. Siswa harus menjawab dengan kalimat, sehingga jawaban tersebut dapat bervariasi berdasarkan pemikiran mereka. Salah satu kelemahan dari jawaban dalam bentuk esei adalah sulitnya menilai jawaban, dan cenderung memakan waktu. Saat ini grading esei otomatis, suatu cara yang dapat mempercepat penilaian esei, sedang banyak dikembangkan. Metode yang digunakan beraneka ragam, salah satunya adalah metode Latent Semantic Analysis (LSA). Metode LSA adalah metode untuk mengekstrak dan merepresentasikan kalimat dalam bentuk matematik atau statistik dari teks dengan jumlah kata yang relatif besar. Nilai dari jawaban esei diperhitungan, dengan mencocokkan ada atau tidak adanya kata yang dianggap penting, pada kelompok kata dalam matriks yang telah dipersiapkan oleh human rater (penilai manusia). Pada makalah ini diuraikan upaya penelitian dan pengembangan metoda LSA, yang dilengkapi dengan pembobotan kata, urutan kata dan persamaan kata untuk meningkatkan ketelitian penilaian esei. Sistim ini diberi nama SIMPLE. SIMPLE digunakan untuk menilai jawaban ujian dalam bahasa Indonesia. Ujian dilakukan secara on-line, melalui web. Dari hasil ujicoba yang telah dilakukan, pada kelas kecil diperoleh nilai kesesuaian dengan human raters berkisar 69.80 % – 94.64 %, sedangkan pada kelas menengah diperoleh nilai berkisar 77.18 % – 98.42 %. Hasil-hasil ini setara dengan hasil metoda grading otomatis LSA terdahulu, yang melakukan penilaian terhadap jawaban ujian dalam bahasa Inggris.

References

  1. Salvatore Valenti, Francesca Neri, Alessandro Cucchiarelli, “An Overview of Current Research on Automated Esei Grading”, Journal of Information Technology Education, Vol. 2, 2003.
  2. Derek Stokley, “E-learning Definition and Explanation (Elearning, Online Training, Online Learning”,http://derekstockley.com.au/elearningdefinition.html, Australia, last up date: 26/02/06.
  3. Laurence Rudner, Phill Gagne, “an Overview of Three Approaches to Scoring Written Eseis by Computer”, Practical Assessment, Research & Evaluation, A peer-reviewed elec. Journal, 2001; http://ericae.net/pare/getvn.asp?v=7&n=26
  4. Lawrence M Rudner, Tahung Liang, “ Automated Esei Scoring Using Bayes’ Theorem”, The Journal of Technology, Learning and Assessment, volume 1, Number 2, June 2002.
  5. Asmawi Zainul, Noebi Nasution, “Penilaian Hasil Belajar”, PAU untuk peningkatan dan Pengembangan Aktivitas Instruksional Ditjen Dikti, DepDiknas, Jakarta, 2001.
  6. Linlin Irene Chen, Ruth Robbins, “On Line Testing for Business Programming Class“, Proc. of 8th Annual International Distance Education Conf. January 23-26, 2001.
  7. Anak Agung Putri Ratna, Luhur Bayuaji & Muhammad Salman, ”Design And Implementation of Distance Learning in Basic Computer Course”, IMSA’01 IASTED Procs., pp. 51-55, Hawaii, 2001.
  8. Anak Agung Putri Ratna, Adjie Pamungkas, Natalia Evianti and Muhammad Salman, ”Daskom On-Line: User Management Implementation on Web Based Learning Application”, Web Net Proceedings, pp 1041-1042, Orlando 2001.
  9. Anak Agung Putri Ratna, Astha Ekadiyanto, Djoko Hartanto, Seinosuke Narita, "Daskom On-Line: Implementation Distance Learning on Basic of Computer Course", Web Net Proceedings, pp 1600-1602, Denver 2002.
  10. Anak Agung Putri Ratna, Patar P Raymont, Natalia Evianti, Djoko Hartanto, Seinosuke Narita, “Distance E-Learning Implementation and Analysis on Jarkom On-line Evaluation System”, IMSA’03 IASTED Proceeding, Hawaii, 2003.
  11. Cambridge University Press, Program Copyright © 1988-1992. Numerical Recipes in C: The Art of Computing. http://www.nr.com.
  12. Eric W Weisstein, “Frobenius Norm”, Mathword, A Wolfram Web Resource, Wolfram Research Inc., 2004, http://www.mathword.wolfram.com/Frobenius/Norm.html. akses tgl 09/09/04.
  13. Thomas Hofmann, “Matrix Decomposition Methods in Information Retrieval”, Presentation Material, KNeuroCOLT Workshop, April 30thMay 2nd 2001, Cumberland Lodge.
  14. Peter W Folz, Sara Gilliam dan Scott A Kendall, “Supporting content-based feedback in online writing evaluation with LSA”, Interactive Learning Environments 8(2): 111-129, New Mexico City University, 2000.
  15. David A. Frisbie, Kristie K. Waltman, Developing a Personal Grading Plan”, Educational Measurement: Issues and Practice, Fall 1992, UW’s Faculty Resource On Grading, Iowa, http://depts.washington.edu/grading/plan/frisbie1.h tm, akses 03/06.
  16. Christian M. Reiner, Timothy W. Bothell, Richard R. Sudweeks, Bud Wood. “Preparing Effective Esei Questions”, new Forums Press, 2002, http://testing.byu.edu/info/handbooks/WritingEffe ctiveEseiQuestions.pdf, akses 03/06.

Download

Share

COinS
 
 

To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.