The evaluation of forms of assessment using n-dimensional filtering.

The evaluation of forms of assessment using n-dimensional filtering. This article describes an unusual approach to the evaluation of theeffectiveness of assessed coursework courseworkNounwork done by a student and assessed as part of an educational courseNoun 1. coursework - work assigned to and done by a student during a course of study; usually it is evaluated as part of the student's provided for a group of graduatestudents at the University of Brighton The University of Brighton (formerly Brighton Polytechnic until its re-designation in 1992) is a multi-site university based in the city of Brighton & Hove (England). , UK. It makes use of metadataexposed and created by the students to discover common values andattitudes to the work that they are required to perform. These metadataare the product of a Web-based system known as CoFIND (CollaborativeFilter in N Dimensions). ********** About CoFIND CoFIND has been described elsewhere (Dron, Mitchell, Boyne, &Siviter, 2000b; Dron, Mitchell, & Boyne, 2002; Dron, Mitchell,Siviter, & Boyne, 1999) and this article presents a relatively briefdescription of it here by way of foundation. The main focus of thisstudy is to examine the different kinds of metadata generated by thestudents and to draw some conclusions about how this relates to theirperceptions of and attitude towards assessment tasks. CoFIND is a self-organizing resource database for learners.Learners enter resources, usually those that can be referred to by anURL URLin full Uniform Resource LocatorAddress of a resource on the Internet. The resource can be any type of file stored on a server, such as a Web page, a text file, a graphics file, or an application program. (although the system can also take free text entries describing anyobject or which are, themselves, resources), into the system. A learnermay have created a resource or may have found it useful and wished toshare it. Learners are encouraged to classify resources using existingclassifications (following conventional usage, these are described as"topics") or new ones which they enter themselves,collaboratively giving the system a functionality not dissimilar to thatof a Web directory such as Yahoo! or Newhoo. Alternatively, it may bethought of as a collaboratively generated set of bookmarks. QUALITIES One of CoFIND's unique features is its use of a form ofmetadata, which we characterize as "qualities" Qualities arethose aspects of resources which learners consider important whenchoosing to use them. Thus, they are statements of value about aresource and might typically consist of words and phrases Words and Phrases?A multivolume set of law books published by West Group containing thousands of judicial definitions of words and phrases, arranged alphabetically, from 1658 to the present. like"good for beginners," "funny,""comprehensive" or "simple." Qualities are used togive ratings to resources which, in turn, provide potentially richer andsubtler indications of the value of a resource than those of othercollaborative rating systems or collaborative filtering Also known as "social filtering" and "social information filtering," it refers to techniques that identify information a user might be interested in. There are different kinds of algorithms used, but the basic principle is to develop a rating system for matching incoming material. engines such asNetPerceptions (www.netperceptions.com), DBLens(dblens.sourceforge.net), or MatchPoint (www.triplehop.com). Qualitiesare then dimensions of CoFIND, providing a multi-faceted collaborativeview of the system relevant to and generated by its users. Through a combination of metrics metricsManaged care A popular term for standards by which the quality of a product, service, or outcome of a particular form of Pt management is evaluated. See TQM. based on the way that usersinteract with the system, including ratings, usage, and the frequencywith which the system is used, CoFIND arranges the presentation oftopics, qualities and resources dynamically, making use of variousvisual cues such as size and position on the page. It is designed torespond to individual local interactions, such as rating a resource orselecting a quality, to produce a self-organized system without acentral designer. CoFIND has a range of different uses, including helping learners tofind appropriate Web-based resources for their learning needs (Dron,Boyne, & Mitchell, 2001; Dron, Mitchell, Boyne, & Siviter,2000a, 2000c; Dron et al., 1999) and collaboratively discovering whatmakes a website good (Dron, Mitchell, Boyne, & Siviter, 2000d).Other uses have been explored or envisaged, such as a gallery whichallows collaborative classification of images, identifying metadata forschema such as IMS (1) See IP Multimedia Subsystem.(2) (Information Management System) An early IBM hierarchical DBMS for IBM mainframes. IMS was widely implemented throughout the 1970s under MVS and continues to be used under z/OS. and as a learning object repository. The studiesdiscussed in this article describe the use of CoFIND to enrich theprocess of student assessment, a very interesting side effect of whichis to allow us to find out some useful information about how studentsperceive the work that they are asked to perform. A LEARNING ECOLOGY CoFIND's qualities are used to rate and discover resourcesthat are not only appropriately classified as belonging to a particulartopic, but which fulfill specified needs described by the qualities. Forinstance, if a learner feels that a resource is good for beginners, thenusing the quality "good for beginners" is not only making astatement about the resource, but also about the learner'sperception of a need for resources which are good for beginners.Resources classified as belonging to a particular topic are returned ina list ordered by their ratings for a selected quality. As ratings aregiven to a resource, so its position in the list of resources for thatparticular combination of topic and quality changes. The average of allratings given for a resource is used to supply its weighting, with asmall bias in favor of resources which have received many ratings. Theeffect of this bias is to preferentially differentiate frequently ratedfrom infrequently in��fre��quent?adj.1. Not occurring regularly; occasional or rare: an infrequent guest.2. rated resources, where their average weighting isotherwise similar. [FIGURE 1 OMITTED] Learners may add new qualities if those which are already availabledo not seem appropriate to their needs. Unfortunately, a newly addedquality will not have been used to rate any resources and so will be oflittle use in retrieving useful lists of resources. This could lead to avicious circle A Vicious Circle (1996) is a novel by Amanda Craig which dissects and satirizes contemporary British society. In particular, it describes the world of publishing -- its aspiring young authors, busy agents and opportunist literary critics. , an example of the cold start phenomenon which affectsmost collaborative filtering systems (Harney, 2000). The phenomenonoccurs because the system only becomes useful once sufficient ratingshave been given, but ratings are only given if the system is used, whichis unlikely because there are insufficient ratings to make it useful. Tohelp overcome this, when adding a new quality, a learner may allow it toinherit To receive property according to the state laws of intestate succession from a decedent who has failed to execute a valid will, or, where the term is applied in a more general sense, to receive the property of a decedent by will. inheritv. ratings from one which already exists. For example, a useradding the quality "simple" may allow it to inherit votes from"good for beginners." Like much of what underlies CoFIND, thisborrows heavily from mechanisms which drive evolution in naturalsystems. In effect, this provides inheritance with variation, allowingcompetition to develop and thus generating qualities which are bestsuited to their environment, i.e., the group of learners who use them. These qualities can appear as a list of hyperlinks, potentially invarious font sizes, dynamically ordered. Figure 1 shows one such list. The size of each quality is proportional to the number of times thequality has been used to rate resources within the selected topic,compared with other qualities. Larger quality labels imply that moreuseful results will be found if the learner follows those links. Theorder of qualities in the list is partly determined by use (qualitieswhich are most frequently selected), partly by votes cast using thosequalities to rate resources within the selected topic, and partly by aweighting based mainly on the age of the quality. To cater for apotentially large number of qualities of low value to other learners,qualities may "die," dropping off the list entirely. Thisprevents inappropriate qualities from retaining a foothold foot��hold?n.1. A place providing support for the foot in climbing or standing.2. A firm or secure position that provides a base for further advancement.footholdNoun1. inCoFIND's ecology. Qualities of little use get smaller, drop downthe list and eventually disappear. Again, this process is inspired bynatural evolutionary processes. The result is a set of qualities in anorder which has adapted to the specific needs of the group of learnerswho created it in a particular topic. STIGMERGY The use of varying font sizes and dynamic ordering of the returnedlist is designed to make use of a positive feedback mechanism foundwidely in natural and human systems known as "stigmergy," akind of indirect communication via the environment (Heylighen, 1999).Examples include the formation of ant trails, the construction oftermite termiteor white ant,common name for a soft-bodied social insect of the order Isoptera. Termites are easily distinguished from ants by comparison of the base of the abdomen, which is broadly joined to the thorax in termites; in ants, there is mounds or the emergence of forest footpaths. In stigmergicsystems, each individual's actions affect its environment which inturn influences the actions of others. For example, ants returning to anest with food leave a trail of pheromones pheromones,any of a variety of substances, secreted by many animal species, that alter the behavior of individuals of the same species. Sex attractant pheromones, secreted by a male or female to attract the opposite sex, are widespread among insects. . Ants that would otherwisewander aimlessly aim��less?adj.Devoid of direction or purpose.aimless��ly adv.aim are more likely to follow a trail of pheromones; thestronger the trail, the greater the likelihood that others will followit. As more ants return with food, the trail gets stronger. When thefood eventually runs out, the scent dissipates and the trail vanishes.In a similar process, termites build their mounds by dropping smalllumps of mud infused with pheromones. The stronger the scent ofpheromones, the more inclined they are to drop mud nearby, thus leadingto clumps clump?n.1. A clustered mass; a lump: clumps of soil.2. A thick grouping, as of trees or bushes.3. A heavy dull sound; a thud.v. , then columns which grow towards each other so as to formcathedral-like structures. In all such systems, order and control is anemergent emergent/emer��gent/ (e-mer��jent)1. coming out from a cavity or other part.2. pertaining to an emergency.emergent1. coming out from a cavity or other part.2. coming on suddenly. property of the system and is not dictated from above. WithinCoFIND, learners are more likely to select qualities in a larger fontand/or higher up the list. Likewise, because learners are more likely to visit resources whichare highly rated, they are more likely to provided ratings for resourceswhich have already been rated, which (if the resources are consideredworthwhile) means that the position of qualities on the list and theirsize is further enhanced. Success breeds success. The polarization polarizationProperty of certain types of electromagnetic radiation in which the direction and magnitude of the vibrating electric field are related in a specified way. thatresults from this is an intentional product of the system, which bothinfluences and is influenced by its users. For a new topic/qualitycombination to gain an effective foothold, it needs to add significantvalue or those qualities which are already successful will overwhelm o��ver��whelm?tr.v. o��ver��whelmed, o��ver��whelm��ing, o��ver��whelms1. To surge over and submerge; engulf: waves overwhelming the rocky shoreline.2. a. it. Self-organization Without constraint, the positive feedback provided by stigmergycould lead to what complexity theorists describe as a Stalinist regime(Kauffman, 1995), a system locked into a fixed pattern of behaviourwhere novelty has no chance to take hold. It is equally undesirable toreach what the same complexity theorists call a Red Queen Regime, wherechange is so frequent that no stability is ever reached and the systemdescends into chaos. CoFIND's algorithms are tuned to create adynamic system balanced at the edge of chaos For the computer game, see .The phrase edge of chaos was coined by computer scientist Christopher Langton in 1990. The phrase originally refers to an area in the range of a variable, λ (lambda), which was varied while examining the , with sufficient stabilityto create structure but with sufficient adaptability to cater forlearners' changing needs. Driven by the positive forces oflearners' needs and stigmergy, evolutionary forces of competitionact on the results, constraining con��strain?tr.v. con��strained, con��strain��ing, con��strains1. To compel by physical, moral, or circumstantial force; oblige: felt constrained to object.See Synonyms at force.2. growth through the survival of thefittest. This can result in a self-organized learning ecology ofpotentially high complexity and variegation VariegationPatchy variation in color.Mentioned in: Malignant Melanoma , with different species ofquality fitting different ecological niches Noun 1. ecological niche - (ecology) the status of an organism within its environment and community (affecting its survival as a species)nichebionomics, environmental science, ecology - the branch of biology concerned with the relations between organisms which relate to thedifferent and contextually situated needs of learners and the varioustopics they explore. Through their combined individual interactions, aclear structure to the available resources emerges without theintercession intercession,n a prayer in which a request is made on behalf of another person. of a teacher or instructional designer. The use of topics in the studies which this article describes wasintentionally constrained con��strain?tr.v. con��strained, con��strain��ing, con��strains1. To compel by physical, moral, or circumstantial force; oblige: felt constrained to object.See Synonyms at force.2. so the mechanism will not be discussed indetail here. In normal use, topics are generated by students and areself-organized using stigmergic and evolutionary processes to help toprovide a guided path through a subject area. This has been discussedelsewhere (Dron et al., 2001; Dron et al., 2000a). Through the use of qualities, CoFIND provides a unique glimpse of agroup of learners' view of what is important in a learningresource. It becomes a mirror of the needs, aspirations aspirationsnpl → aspiraciones fpl(= ambition); ambici��n faspirationsnpl (= hopes, ambition) → aspirations fpland motivationof the learners who use it, whatever these may be. In some sense, CoFINDmay be thought of as a reflection of a group mind, a learning landscapeor an ecology which has evolved around its users. It is this aspect ofthe system which the following two case studies use. DESCRIPTION OF THE CASE STUDIES Both of these studies involved the same group of 35 studentsstudying for an intensive one-year conversion Masters degree inInformation Systems. All of the students had degrees in subjectsunrelated to Information Systems and can therefore be considered asmature learners. The intensity of the course and its group-orientatedethos typically results in a very close-knit community of highlymotivated individuals. Both studies presented here relate to a stream of the course thatteaches networking and Internet technologies, from low-end protocolsthrough to Web application building. The networking and Internettechnologies stream straddles two semesters. At the end of each semester se��mes��ter?n.One of two divisions of 15 to 18 weeks each of an academic year.[German, from Latin (cursus) s is a traditional unseen examination In the UK, an unseen examination is an essay test in school or college, where the student does not know what questions are going to be asked. The student is required to answer questions based upon what they have learned over the course of their academic study. preceded by a two-week personalassignment. These studies primarily look at the use of CoFIND as asupport and evaluation mechanism for the work produced in these personalassignments. CoFIND is designed to capture the patterns of use of itscommunity. The number of ratings and uses of different qualities andtopics were gathered, including information about which students hadgiven which ratings. These were filtered to remove access by the tutor,who had participated only by viewing the pages. To ensure that theresults were not skewed skewedcurve of a usually unimodal distribution with one tail drawn out more than the other and the median will lie above or below the mean.skewedEpidemiology adjective Referring to an asymmetrical distribution of a population or of data , the tutor viewed each topic and quality anequal number of times, thus preventing his own contribution frommaterially affecting the stigmergic effects of the system. Briefinformal interviews were conducted to explore anomalies that arose inthe course of the studies. THE FIRST ASSIGNMENT The first assignment given at the end of the first semesterrequired students to produce Web pages written using XHTML (EXtensible HTML) A markup language for Web pages from the W3C. XHTML combines HTML and XML into a single format (HTML 4.0 and XML 1.0). Like XML, XHTML can be extended with proprietary tags. Also like XML, XHTML must be coded more rigorously than HTML. to providetutorials on technologies related to the module. The students wereallowed to choose any network-related topic which interested themincluding, but not limited to, any topic taught on the stream. For some,this provided an opportunity to revise or clarify work already covered,which was therefore of interest to the entire cohort and would beperceived as useful preparation for the Information Systems and cantherefore be considered as mature learners. For others, it was anopportunity to explore areas on which they were writing a 12,000 worddissertation dis��ser��ta��tion?n.A lengthy, formal treatise, especially one written by a candidate for the doctoral degree at a university; a thesis.dissertationNoun1. due in the second semester. All students selected a topicrelated to one of these two functions. Given the intensive nature of thecourse, this is to be expected, as the students needed to make the bestuse of the time available to achieve their main goal of gaining aMasters of Science and most did not wish to investigate a topic out ofsimple curiosity. At the end of the assignment, the students were required to addreferences to their pages to CoFIND. They were also required to look atand rate at least three of their colleagues' tutorials, with asmall penalty (a total of three percent of the marks) being applied tothose who failed to do so. There was no need to implement this penalty,as all students rated more than three sites. Following discussions withthe course leader it was decided that the students' own assessmentsof each other should not contribute to the formal mark received. The students were already familiar with CoFIND, as it had acted asa resource base since the start of the stream, providing references boththey and the tutor had found and rated. It is significant that althoughall students used the system, leading to a database of nearly 100relevant rated resources, the level of activity and especially of ratinghad not been very high, averaging around 10 ratings per week over thecourse of the semester. THE SECOND ASSIGNMENT The second assignment given at the end of the second semesterrequired students to produce a Web-based application See Web application. implementing someof the functionality of a system which they had already been analysing,designing and programming for other streams of the course, based on arich case study designed to give a moderately authentic experience ofworking on an IT project. Crucially, the assignment was thereforeperceived as an activity giving summary evidence of the skills whichthey had been learning during the latter half of the stream. Theassignment was based on a fictitious Based upon a fabrication or pretense.A fictitious name is an assumed name that differs from an individual's actual name. A fictitious action is a lawsuit brought not for the adjudication of an actual controversy between the parties but merely for the purpose of company called Stowaways Stowaways are a Portuguese band from Matosinhos, who formed in 2001. They are made up of Nuno Sousa (vocals and guitar); Pedro Gon?alves (guitar); Jo?o Carujo, (drums)and S��rgio Seabra (bass). Fred on keyboards and Jo?o Covita on the accordion are more recent additions. offeringholidays on the rivers and canals of England. Like the first assignment, students were required to enter links totheir work into the CoFIND system with the threat of marks beingdeducted de��duct?v. de��duct��ed, de��duct��ing, de��ductsv.tr.1. To take away (a quantity) from another; subtract.2. To derive by deduction; deduce.v.intr. from students for failing to rate at least two of theircolleagues' sites. Like the first assignment, here was no need toapply this penalty, although most students only rated two sites. Again,the course leader advised against allowing the students' ownassessments to contribute to their formal mark for the module. RESULTS Over three times as many ratings were given for the firstassignment as for the second (288 for the first, 88 for the second).This did not seem to be unduly affected by other factors such aslearning or boredom BoredomSee also Futility.Aldegonde, Lord St.bored nobleman, empty of pursuits. [Br. Lit.: Lothair]Baudelaire, Charles(1821–1867) French poet whose dissipated lifestyle led to inner despair. [Fr. Lit. as, apart from during the period of the assignmentsand troughs during vacation periods, use of the system for sharingresources remained consistent throughout both semesters. As may be seen from Figure 2, there is very little similaritybetween the uses of qualities in the two assignments. Although newqualities were available and were added during the second assignment,all of the existing qualities used in the first assignment had survivedthe evolutionary process and were still available for the students touse. A closer look reveals that, in order of popularity, the top fivequalities for the first assignment were "useful,""comprehensive," "simple," "good forbeginners" and "good overview." Three of these (includingthe top two) were not used at all for rating work on the secondassignment, and even those that were--"good for beginners" and"simple--achieved less than five percent of the votes cast. Bycontrast, the top five qualities for the second (Stowaways) assignmentwere "invokes that holiday feeling," "good," "avaliant VALIANTValsartan in Acute Myocardial Infarction Trial Cardiology A series of multinational M&M trials to determine the effects of valsartan–Diovan® attempt," "floats like a brick," "good forbeginners" and "simple" (joint fifth place). Bothassignments showed the traditional bell curve in grades achieved,without any clear difference in the distribution of marks. DISCUSSION Qualities as Aids to Learning The qualities used by the students for the first assignment are,broadly, those that would be helpful in finding resources from which tolearn and to revise for exams. They are the sort of words which ateacher might use to describe resources from which a student mightlearn, whether explicitly or implicitly. If a teacher suggests thatstudents read a particular resource it is by definition because he orshe feels that the students would find it useful (the most popularquality for the first assignment). The teacher might then go on toexplain why it is useful (for instance, it gives a comprehensive view ofthe subject, it is simple, good for beginners or it gives a goodoverview) which would help to match that resource to the needs of aparticular student or group of students. One of the charms of the CoFINDsystem is that the students themselves are likely to have a better ideaof how particular resources match their own needs than the teacher, atleast when averaged, and the use of qualities in the first assignmentseems to be a clear expression of those needs. By contrast, with the exception of the last two qualities in jointfifth place ("simple" and "good for beginners"),there is nothing in the qualities for the second assignment whichsuggests any value at all as a learning resource. Instead, the qualitiesused are variants on the single dimension of good to not-good which isused in virtually all collaborative rating systems on the Web. In fact,interviews with students indicate that the last two qualities are notexceptions at all, but were supplied by the overly modest students whoadded the resources themselves, not as a description of theireducational value but as a way of drawing attention to their inadequacy.Given that the second assignment was based around holidays on waterways The list of waterways is a link page for any river, canal, estuary or firth. International waterwaysDanish straits Great Belt Oresund Bosporus Dardanelles ,the fact that two of the most popular qualities were humorous variationson the theme is noteworthy, again giving clear evidence that thestudents were not concerned with the outcome of their voting, merelythat it was seen to be done. Term Ambiguity Term ambiguity, of the sort seen in the differing uses of"good for beginners" and "simple," is a difficultproblem which affects any classification system (Lakoff, 1987). It isprimarily for this reason that CoFIND has been targeted squarely atsmall, tightly focussed cohorts of learners with clear and well-definedneeds, on the assumption that they would, in general, share a vocabularyand worldview world��view?n. In both senses also called Weltanschauung.1. The overall perspective from which one sees and interprets the world.2. A collection of beliefs about life and the universe held by an individual or a group. . As this study shows, even within such a constrainedenvironment such worldviews are dynamic and transient things, shiftingwith the needs of the learners. Learning changes people and thequalities which they choose to assess learning resources provide a clearindication of this. How this affects the way that learners laterperceive resources rated using an earlier world view has yet to beshown, but will make an interesting subject for further study. Frequency of Rating The relevance to the students of the output of the two assignmentsis clearly demonstrated in the number of ratings given in each instance.The fact that the results of the second assignment were of no apparentfurther use resulted in the minimum of ratings being given, drivenalmost entirely by the extrinsic EVIDENCE, EXTRINSIC. External evidence, or that which is not contained in the body of an agreement, contract, and the like. 2. It is a general rule that extrinsic evidence cannot be admitted to contradict, explain, vary or change the terms of a contract or of a motivation of not wanting to lose anymarks, but without the intrinsic motivation provided by the firstassignment of wanting to help to build a collaborative resource offurther use to the learning needs of the cohort. [FIGURE 2 OMITTED] Justifications for assessment are often given on the grounds thatthey help to provide focus and motivation for students, as well asproviding a basis for much of the formative formative/for��ma��tive/ (for��mah-tiv) concerned in the origination and development of an organism, part, or tissue. process essential tolearning (Bruner, 1966; Gagne, 1985). They are also an interface betweenexamining institutions and the outside world, giving an indication of astudent's worth to potential future employers. However, at the sametime, they are often obstacles to learning parts of a system which feedson itself at the expense of the needs of its users. Their primarymotivation, it is hoped, would be to learn, rather than to achieve aqualification (Holt, 1977; Illich, 1971; Whitehead whitehead/white��head/ (hwit��hed)1. milium.2. closed comedo.white��headn.1. , 1929). Thesefindings provide some evidence that, when used purely as an extrinsicmotivator, assessments are far less significant to the students thanwhen those assessments contribute directly to that learning process.This is entirely in keeping with the model of engaged learning proposedby Jones, Valdez, Nowakowski, and Rasmussen (1995), who state firmlythat assessment should involve a performance or demonstration, usuallyfor a real audience and useful purpose, as well as having meaning andpurpose for the learner. In this case there is a second layer ofindirection Not direct. Indirection provides a way of accessing instructions, routines and objects when their physical location is constantly changing. The initial routine points to some place, and, using hardware and/or software, that place points to some other place. at work, inasmuch as in��as��much as?conj.1. Because of the fact that; since.2. To the extent that; insofar as.inasmuch asconj1. since; because2. from the students' perspective themost useful function served by the results of the assessed task in thefirst assessment was to feed their needs in addressing a furtherassessment (the end-of-semester examination). Methodological Weaknesses Though interesting, the results generated by CoFIND are not inthemselves conclusive. This was not set up as a controlled experiment "Controlled Experiment" is an episode of the original The Outer Limits television show. It first aired on 13 January, 1964, during the first season. IntroductionA martian controller is assigned to investigate the phenomenon of murder on Earth. ,but came as a serendipitous ser��en��dip��i��ty?n. pl. ser��en��dip��i��ties1. The faculty of making fortunate discoveries by accident.2. The fact or occurrence of such discoveries.3. An instance of making such a discovery. finding from an ongoing process of actionresearch into how educationally useful self-organization can beencouraged through the use of a piece of collaborative software This is a list of collaborative software (or list of groupware) applications. Wiki software is on a list of wiki software. Open source or free softwareThe following are open source or free software applications. . Thetentative conclusions drawn are mainly extrapolations from the datacollected, which provide useful evidence but not proof of the case.Although care was taken to ensure that the group was not influenceddifferently from one use to the next, there are several points at whichthe results could have been skewed and where the interpretations givenmay be faulty, especially those relating to relating torelate prep → concernantrelating torelate prep → bez��glich +gen, mit Bezug auf +accthe number of ratings givenoverall. The following possibilities seem particularly relevant: * The first study resulted in more ratings as a result of aninitial bout of enthusiasm by the students who, by the time of thesecond study, had decided that CoFIND was not as useful as they had atfirst supposed. From the informal follow-up conversations with selectedstudents and from unsolicited un��so��lic��it��ed?adj.Not looked for or requested; unsought: an unsolicited manuscript; unsolicited opinions.unsolicitedAdjective comments in general course evaluations A course evaluation is a paper or electronic questionnaire, which requires a written or selected response answer to a series of questions in order to evaluate the instruction of a given course. this did not seem to be the case, but the students may well have givenpositive evaluations in the hope of pleasing the tutor; * The reduced number of ratings given during the second assignmentwas partly the result of greater perceived stress due to the impending im��pend?intr.v. im��pend��ed, im��pend��ing, im��pends1. To be about to occur: Her retirement is impending.2. end of the taught part of the course. However, the actual workload ofthe students at the time of the first study was somewhat higher, so thiseffect was probably counterbalanced coun��ter��bal��ance?n.1. A force or influence equally counteracting another.2. A weight that acts to balance another; a counterpoise or counterweight.tr.v. ; * The reduced number of ratings given during the second assignmentwas because the enthusiasm of the tutor for the system had waned, thusnot influencing the students as strongly as during the first. Althoughthe tutor was not aware of any loss of enthusiasm and care was taken toavoid any significant differences in the presentation of the task, itwould not be possible to guard against tacit cues perhaps brought on bydiffering workloads or other external influences; * CoFIND slowed down perceptibly per��cep��ti��ble?adj.Capable of being perceived by the senses or the mind: perceptible sounds in the night.[Late Latin perceptibilis, from Latin perceptus as more qualities and resourceswere added. This might well have reduced the inclination of users toprovide ratings for the later assignment, but no evidence was found forthis one way or the other; * The feedback from the tutor resulting from the first study hadnot been what was expected and, therefore, the students lost motivation.The tutor actively avoided giving any feedback at all on the way thatqualities had been used and this may have been perceived negatively.This possibility was considered unlikely and not pursued further, but inretrospect may have adversely influenced the number of ratings given; * The students had become more sophisticated users of the system bythe time of the second study and no longer saw the value in usingpedagogically ped��a��gog��ic? also ped��a��gog��i��caladj.1. Of, relating to, or characteristic of pedagogy.2. Characterized by pedantic formality: a haughty, pedagogic manner. useful qualities. Feedback from the interview and fromunsolicited comments during the course evaluation suggests this was notthe case but this possibility was not rigorously pursued. * Although similar patterns of qualities have been seen in severalinstances of CoFIND including those used with level 2 undergraduates,the strong differences observed between the uses of qualities in the twoassignments may at least partly be influenced by the fact that thesestudents were sophisticated and focused postgraduate adult learners Adult learner is a term used to describe any person socially accepted as an adult who is in a learning process, whether it is formal education, informal learning, or corporate-sponsored learning. . Asmall number of these had previously been in the teaching profession andmight therefore be seen as even more reflective learners. In the contextof building a self-organizing learning environment, this is a positivething, as such skills and interests might contribute effectively to thediversity and richness of the system, but from the point of view of thisstudy it was not taken into consideration and may have influenced theway that qualities developed. This is not in itself a serious flaw, butillustrates the context sensitivity brought about by the way that CoFINDshapes itself to its users. A more thorough study is planned for the coming year using a newer,more finely tuned, better performing version of CoFIND. This willinclude a closer analysis of individual contributions which will berelated to a series of pre- and post-study questions intended toidentify in greater depth the benefits or otherwise that the studentsperceive from the use of CoFIND. It is also hoped to correlate theresults using different groups of students. Possibilities It is possible to imagine a scenario where CoFIND might be used tocollaboratively generate an entire course in higher education higher educationStudy beyond the level of secondary education. Institutions of higher education include not only colleges and universities but also professional schools in such fields as law, theology, medicine, business, music, and art. . Thecourse could be constructed from a set of assessments, each building onthe last, each leading to collaboratively generated further knowledgewith direct relevance to the next task. More controversially, it ispossible that such a system might be formed without the directintercession of a teacher in the design process. If students were to begiven sufficient resources (including static materials, reactive supportfrom subject experts and suitable infrastructure for communication) andaimed very roughly in the right direction, the most highly rated studentresources could provide the input for the next wave of assessment. Ateacher or other arbiter (perhaps the students themselves) could take upthe most popular themes, topics and areas and set them as the focus ofanother CoFIND-mediated investigation. The process would continue untilthe students felt that they had learned enough or an external arbiter(with full access to student work and opinions about it through CoFIND)considered that learning had been demonstrated. In this way, aself-organized system of learning might be formed, with learningresources (being generated by the students themselves) naturally fallingwithin Vygotsky's zones of proximal proximal/prox��i��mal/ (-mil) nearest to a point of reference, as to a center or median line or to the point of attachment or origin. prox��i��maladj. development (Vygotsky, 1978).The approach would best be suited to a subject with a large body ofknowledge mapped to a loose and flexible body of theory, as found in themedical and computing professions, where knowledge is achieved as agestalt Gestalt(gəshtält`)[Ger.,=form], school of psychology that interprets phenomena as organized wholes rather than as aggregates of distinct parts, maintaining that the whole is greater than the sum of its parts. more than being assembled systematically from axioms This is a list of axioms as that term is understood in mathematics, by Wikipedia page. In epistemology, the word axiom is understood differently; see axiom and self-evidence. Individual axioms are almost always part of a larger axiomatic system. . CONCLUSION CoFIND is intended as a tool to aid collaborative learning Collaborative learning is an umbrella term for a variety of approaches in education that involve joint intellectual effort by students or students and teachers. Collaborative learning refers to methodologies and environments in which learners engage in a common task in which each , but itis clear from the example of these two small studies that in order forit to be effective, it needs to be considered as part of a largersystem. Simple application of technology is not enough to bring aboutuseful collaboration, even where that technology is explicitly designedto replace some of the mediation role of traditional teaching. CoFIND has shown itself to be a useful research tool that can forman image of the collective attitude of a group of students towards abody of work. In these studies, CoFIND has helped to bring the groupattitudes towards assessment tasks into sharp relief, demonstratingsomething which should have been clear already, that authentic tasksdesigned which have applicability to students' needs are morelikely to be useful to them than those which do not. Viewed fromstudents' own perspectives, such needs may be quite different fromthose which teachers believe them to be. The second case study was awell-intentioned and carefully designed integrated assignment providingwhat the teaching staff saw as an authentic grounding in the subject ofstudy. However, the system of assessment and examination can control anddetermine the students' priorities and goals far more than thepromise of some distant set of learning objectives (Dewey, 1897). Thesuccess of the first assignment lies not in the success of the studentsin achieving the desired learning objectives but in its relevance totheir immediate needs, consequently providing greater long-term valuethan that of the second. With students benefiting from each other'swork, gaining the benefits of "teachback" (Pask, 1976) throughexplaining a subject to others all within a framework which reflectstheir own conceptions of what is of value in a learning resource, it ispossible to conceive of Verb 1. conceive of - form a mental image of something that is not present or that is not the case; "Can you conceive of him as the president?"envisage, ideate, imagine systems incorporating tools like CoFIND, wherethe role of the teacher may dissolve further into the background as aninstigator in��sti��gate?tr.v. in��sti��gat��ed, in��sti��gat��ing, in��sti��gates1. To urge on; goad.2. To stir up; foment.[Latin , a helper and a validator rather than a provider ofknowledge. Working examples of CoFIND and further presentations and papers onthe subject which are referred to in this article may be found athttp://www.cofind.net. REFERENCES Bruner, J.S. (1966). Toward a Theory of Instruction. Cambridge, MA:The Belknap Press of Harvard University Press The Harvard University Press is a publishing house, a division of Harvard University, that is highly respected in academic publishing. It was established on January 13, 1913. In 2005, it published 220 new titles. . Dewey, J. (1897). My Pedagogic ped��a��gog��ic? also ped��a��gog��i��caladj.1. Of, relating to, or characteristic of pedagogy.2. Characterized by pedantic formality: a haughty, pedagogic manner. Creed. The School Journal, LIV(3),77-80. Dron, J., Boyne, C., & Mitchell, R. (2001). Footpaths in theStuff Swamp swamp,shallow body of water in a low-lying, poorly drained depression, usually containing abundant plant growth dominated by trees, such as cypress, and high shrubs. . Paper presented at the WebNet 2001, Orlando, Florida The city of Orlando is a major city in central Florida and is the county seat of Orange County, Florida. According to the 2000 census, the city population was 185,951. A 2006 U.S. . Dron, J., Mitchell, R., Boyne, C., & Siviter, P. (2000a).CoFIND: steps towards a self-organising learning environment. Paperpresented at the WebNet 2000, San Antonio, Texas “San Antonio” redirects here. For other uses, see San Antonio (disambiguation).San Antonio is the second most populous city in Texas, the third most populous metropolitan area in Texas, and is the seventh most populous city in the United States. As of the 2006 U.S. . Dron, J., Mitchell, R., Boyne, C., & Siviter, P. (2000b).Darwin among the indices: a report on COFIND, a self-organising resourcebase. Paper presented at the Sixth International ISKO ISKO International Society for Knowledge Organization Conference,Toronto. Dron, J., Mitchell, R., Boyne, C., & Siviter, P. (2000c, May15-19 2000). Towards a Self-Organising Learning Environment. Paperpresented at the WWW9, Amsterdam. Dron, J., Mitchell, R., Boyne, C., & Siviter, P. (2000d).Website Evaluation System: Collaboratively Discovering what makes aWebsite Good. Paper presented at the WebNet 2000, San Antonio, Texas. Dron, J., Mitchell, R., & Boyne, C. W. (2002). EvolvingLearning in the Stuff Swamp. In N. Patel (Ed.), Adaptive EvolutionaryInformation Systems. Hershey, PA: Idea Group. Dron, J., Mitchell, R., Siviter, P., & Boyne, C. (1999).CoFIND- an experiment in n-dimensional collaborative filtering. Paperpresented at the WebNet 99, Honolulu, Hawaii For the city and county of Honolulu, see City & County of Honolulu.“Honolulu” redirects here. For other uses, see Honolulu (disambiguation).Honolulu is the capital as well as the most populous community of the State of Hawaii, United States. . Gagne, R. (1985). The Conditions of Learning (4th ed.). New York New York, state, United StatesNew York,Middle Atlantic state of the United States. It is bordered by Vermont, Massachusetts, Connecticut, and the Atlantic Ocean (E), New Jersey and Pennsylvania (S), Lakes Erie and Ontario and the Canadian province of :Holt, Rhinehart & Winston. Harney, J. (2000). IT self-help. Knowledge Management. Heylighen, F. (1999). Collective Intelligence and itsImplementation on the Web: algorithms to develop a collective mentalmap. Computational and Mathematical Theory of Organizations, 5(3),253-280. Holt, J. (1977). Instead of Education: Ways to help people dothings better. Harmondsworth: Penguin. Illich, I. (1971). Deschooling Society. New York: Harper & Row. Jones, B., Valdez, G., Nowakowski, J., & Rasmussen, C. (1995).Plugging in: Choosing and using educational technology. Washington DC:Council for Educational Research and Development, and North CentralRegional Educational Laboratory. Kauffman, S. (1995). At home in the Universe: The Search for Lawsof Complexity. London: OUP OUP(in Northern Ireland) Official Unionist Party . Lakoff, G. (1987). Women, Fire and Dangerous Things. Chicago: TheUniversity of Chicago Press The University of Chicago Press is the largest university press in the United States. It is operated by the University of Chicago and publishes a wide variety of academic titles, including The Chicago Manual of Style, dozens of academic journals, including . Pask, G. (1976). Conversation Theory--applications in education andepistemology epistemology(ĭpĭs'təmŏl`əjē)[Gr.,=knowledge or science], the branch of philosophy that is directed toward theories of the sources, nature, and limits of knowledge. Since the 17th cent. : Elsevier. Vygotsky, L. S. (1978), Mind in Society. London: Harvard UniversityPress. Whitehead, A.N. (1929). The Aims of Education and Other Essays. NewYork: The Free Press. JON DRON AND CHRIS BOYNE, UNIVERSITY OF BRIGHTON, UK; RICHARDMITCHELL Richard Mitchell may refer to: Richard Charles Mitchell, known as Bob Mitchell (UK politician) (1927–2003), British Labour MP who left to join the Social Democrat Party Richard Mitchell (academic), U.S. , INFERDATA, USA E-MAIL e-mail:see electronic mail. e-mailin full electronic mailMessages and other data exchanged between individuals using computers in a network. : jon.dron@brighton.ac.uk E-MAIL: c.w.boyne@brighton.ac.uk E-MAIL: richard@inferdata.com