{"id":12824,"date":"2015-04-30T21:42:22","date_gmt":"2015-04-30T21:42:22","guid":{"rendered":"http:\/\/rafaelfajardo.com\/portfolio\/locating-the-big-hole-in-hci-research\/"},"modified":"2015-04-30T21:42:22","modified_gmt":"2015-04-30T21:42:22","slug":"locating-the-big-hole-in-hci-research","status":"publish","type":"post","link":"https:\/\/rafaelfajardo.com\/portfolio\/locating-the-big-hole-in-hci-research\/","title":{"rendered":"Locating &#8220;The Big Hole in HCI Research&#8221;"},"content":{"rendered":"<p><a href=\"http:\/\/notesonresearch.tumblr.com\/post\/114677723783\/locating-the-big-hole-in-hci-research\" class=\"tumblr_blog\">notesonresearch<\/a>:<\/p>\n<blockquote>\n<p>At <a href=\"http:\/\/chi2014.acm.org\">CHI 2014<\/a> a paper was presented (<a href=\"http:\/\/www.ee.oulu.fi\/~vassilis\/files\/papers\/chi14.pdf\">Liu et al., 2014<\/a>) which sought to demonstrate, through an analysis of keywords specified in a large tranche of CHI papers (from the last 20 years), that HCI research is lacking in \u201cmotor themes\u201d. According to the paper, a motor theme is a commonly addressed topic in a given academic discipline that defines the research \u201cmainstream\u201d. Motor themes themselves are in turn made from keyword clusters that emerge during a co-word analysis process performed on the collection of keywords, and are found to have particular values of centrality and density. (Note that co-word analysis was initially popularised by Michel Callon and other STS researchers for the study of scientific disciplines, based upon a conceptual backdrop of actor-network theory.) Keyword clusters would be things like \u201ccollaboration\u201d and \u201chandheld devices\u201d amongst many others (see Liu et al. (2014) for the full list), while centrality and density metrics rate \u201chow \u2018central\u2019 a theme is to the whole field\u201d and \u201cthe internal cohesion of the theme\u201d (<a href=\"http:\/\/www.ee.oulu.fi\/~vassilis\/files\/papers\/interactions15.pdf\">Kostakos, 2015<\/a>).<\/p>\n<figure><img decoding=\"async\" src=\"https:\/\/66.media.tumblr.com\/4a9d72eb5709104b2add0fad08240eae\/tumblr_inline_nltv8sbvvx1qe6gmj_500.png\" alt=\"image\" \/><\/figure>\n<p>I attended the conference talk\u2014delivered with panache by <a href=\"http:\/\/www.ee.oulu.fi\/~vassilis\/\">Vassilis Kostakos<\/a>\u2014and the paper received a curiously noisy reception; this was unusual for the normally serene audience at CHI, so I felt it must have touched a nerve for people, resonating with existing concerns some researchers were already occupied by in some way. There were audible gasps when Kostakos produced the punchline of the talk: a graph plotting the apparent absence of keyword clusters in the coveted \u201cQuadrant I\u201d, i.e., the motor theme zone (see image below). This graph was set in opposition to some published co-word analyses of other research areas (e.g., stem cell research, psychology, etc.), which provided more \u2018healthy\u2019 plots.<\/p>\n<figure><img decoding=\"async\" src=\"https:\/\/66.media.tumblr.com\/c9e335a57c1b17881237bc183e9f3e09\/tumblr_inline_nltv9kS5T11qe6gmj_500.png\" alt=\"image\" \/><\/figure>\n<p>The tenor of the talk and the response it received have to be considered alongside other activities appearing at CHI in recent years\u2014for instance, the <a href=\"http:\/\/chi2014.acm.org\/spotlights\/CHI-Science-proposal-final.pdf\">\u201cInteraction Science\u201d SIG of 2014<\/a>\u00a0along with the emergence of <a href=\"http:\/\/www.replichi.com\">yearly events at CHI around \u2018replication\u2019 from 2011 onwards<\/a>. It was this chain of events that made me start to wonder whether a somewhat-dormant cultural undercurrent at CHI (and in HCI more broadly) had been surfaced.<\/p>\n<p>Before I continue I should strongly emphasise that it is extremely important Kostakos and others (e.g., <a href=\"http:\/\/chi2014.acm.org\/spotlights\/CHI-Science-proposal-final.pdf\">Howes et al. (2014)<\/a>, <a href=\"http:\/\/www.replichi.com\/replichi2011.php\">Wilson et al. (2011)<\/a>, etc.) are discussing the \u2018state of the field\u2019. We should applaud them for bringing up this difficult conversation when it is far easier to continue with \u2018business as usual\u2019 and to shy away from mulling over what can be contentious arguments (which HCI and CHI in particular seems to prefer to avoid\u2014often confusing challenges <i>to research<\/i>\u00a0with attacks <i>on researchers<\/i>). While I have very different views on the topic, the emergence of a debate about the very idea of HCI, what it is and its seeming contradictions, seems like a valuable activity for us and probably long overdue. In many ways this reflects some of the preoccupations of early HCI manifest in the exchanges between <a href=\"http:\/\/citeseerx.ist.psu.edu\/viewdoc\/download?doi=10.1.1.22.4849&amp;rep=rep1&amp;type=pdf\">Carroll and Campbell (1985)<\/a>, <a href=\"http:\/\/www.ics.uci.edu\/~redmiles\/inf233-FQ07\/oldpapers\/Prospects.pdf\">Newell and Card (1985)<\/a>, and others.<\/p>\n<p>What follows below is a list of my objections, highlights of what I perceive as confusions, as well as agreements (in a strange way) with Liu et al.\u2019s paper and <a href=\"http:\/\/www.ee.oulu.fi\/~vassilis\/files\/papers\/interactions15.pdf\">Kostakos\u2019s corresponding ACM <i>interactions<\/i> magazine piece<\/a>,\u00a0\u201cThe Big Hole in HCI Research\u201d . There comes with this an implication that the discussion is also applicable to the broader cultural movement that I felt it represents in HCI.<\/p>\n<p><\/p>\n<h2>Low-hanging fruit: On method<\/h2>\n<p>The easiest line of attack in academic work is usually \u2018going for the methods\u2019. Often this is a proxy for other problems that a peer reviewer can\u2019t necessarily articulate. It\u2019s also a strategy reviewers may employ when they have fundamental perspectival differences with the approach taken by the authors of the paper but are perhaps unable to step outside their own perspective for a moment (I would be the first to admit having done this in the past). However, in Liu et al.\u2019s paper, understanding problems with method help us unpack what I see as deeper confusions around disciplinarity and the status of HCI in its relation to \u2018science\u2019 (which I shall relentlessly retain in quotation marks within this post, sorry!). So this is where I start: at the low-hanging fruit.<\/p>\n<p>In principle co-word analysis (of academic papers) clearly has value in providing surveys of particular attributes of publication corpora. Yet we should also exercise some caution here: the <i>claims<\/i> made off the back of the co-word analysis must have attention paid to them, and therefore the <i>basis<\/i>\u00a0upon which such claims are being made. First we should remind ourselves of these claims. The subsequent summary of the original CHI paper (<a href=\"http:\/\/www.ee.oulu.fi\/~vassilis\/files\/papers\/interactions15.pdf\">Kostakos, 2015<\/a>) states that the co-word analysis \u201cconsiders the keywords of papers, how keywords appear together on papers, and how these relationships change over time\u201d. Doing a co-word analysis, it is argued, therefore \u201ccan map the \u2018knowledge\u2019 of a scientific field by considering how concepts are linked\u201d. To clarify, this is indeed a significant claim: firstly that co-word analysis of paper keywords is an adequate method for \u2018mapping knowledge\u2019 for a given discipline, and secondly that it can also be used to demonstrate \u2018gaps\u2019 in a knowledge space.<\/p>\n<p>My problem with this methodologically is that there is no obvious sense in which co-word analysis of the keywords of papers can provide an adequate overview of \u201cthe \u2018knowledge\u2019 of a scientific field\u201d (Kostakos, 2015) because it does not follow that keywords are being deployed by authors in order to provide some set of indices to some known-in-common \u2018map\u2019 of disciplinary knowledge in the first place. Instead one must subscribe to the idea that authors\u2019 deployments of keywords are driven by some \u2018hidden order\u2019 which only becomes visible through the application of the method of co-word analysis. I want to question that claim.<\/p>\n<p>In order to unravel why I might question this, I should explain that I think that some confusions are being made about just what is being \u2018done\u2019 in the writing of keywords for academic papers. These confusions touch on more fundamental misunderstandings about language. Firstly we have to consider how keywords are encountered in the situation in which CHI papers are read (i.e., consider the visual practices of paper-reading CHI-formatted research): they are placed on the first page of the paper, they are prominent under the abstract (where we might start reading) and they have their own headed section. (We don\u2019t \u2018read\u2019 keywords, we \u2018look at them\u2019; I think there is a distinction.) The point is that their visual organisation sets them up to do particular kinds of work for the reader of the paper only <i>as it is read<\/i>, i.e., they cannot be easily removed from their situational relevance to the surrounding text and the manner by which we as readers encounter them.<\/p>\n<p>For instance, authors may deploy keywords according to myriad of possible reasons, many of which may or may not pertain to some \u2018hidden order\u2019\u2014i.e., the \u2018knowledge map\u2019 that is discoverable through co-word analysis. Here are some examples. Keywords may get deployed as terms for <a href=\"http:\/\/portal.acm.org\">ACM Digital Library<\/a> search optimisation. Keywords may be \u2018signals\u2019 for allying oneself to some sub-community of researchers or \u2018sending a message\u2019 to another. Keywords can be used as discriminators of novelty perhaps via the creation of new terms (and thus claiming prospective research spaces). Keywords can be referents to established corpuses of work, framing judgements on the work through the lens of an an existing tradition in order to tell reviewers that \u201cthis is one-of-those-papers, so judge it on those terms\u201d. And, of course still possible, keywords may be indices or intellectual coordinates to some agreed-upon \u2018map\u2019 of HCI knowledge (although one might ask by which textbook those coordinates are even constructed). The point is that the practical purposes of keyword deployment get lost in the co-word analysis because all keywords are treated in the same way.<\/p>\n<p>None of the above necessarily diminishes Kostakos\u2019s notion of \u201cThe Big Hole in HCI\u201d but it certainly exposes some problematic methods by which the claim is substantiated in the first place. Nevertheless, in order to provide a grounding for such a claim, Liu et al. must also conceptualise HCI as a discipline, which is what I turn to next.<\/p>\n<p><\/p>\n<h2>On disciplines and disciplinarity<\/h2>\n<p>Both Liu et al.\u2019s CHI paper and Kostakos\u2019s <i>interactions<\/i>\u00a0article refer to HCI on a number of occasions as a discipline. The main argument refers to a \u2018lifecycle\u2019 for motor themes and their role in disciplines. Disciplinary architecture here is described by various quadrants (see image below), tracking themes as they are born (\u201cQuadrant III: Emerging or declining themes\u201d), begin to stabilise (\u201cQuadrant IV: Basic and transversal themes\u201d), go mainstream (\u201cQuadrant I: Motor themes\u201d) and then die off (back to Quadrant III) or perhaps decline (\u201cQuadrant II: Developed but isolated themes\u201d). Themes may never reach Quadrant I or may go straight to Quadrant II, or perhaps get stuck in Quadrant IV or never make it past Quadrant III. But the basic idea is of the lifecycle and notions of a healthy movement of themes across the graph.<\/p>\n<p>This description, of course, assumes HCI\u2019s classification as a discipline and offers remedial advice for establishing its stability (see the discussion on implications for design below). Largely this assertion passes without comment; in fact this reference to HCI as a discipline is <i>necessary<\/i>\u00a0in order to make sure it becomes comparable with other disciplinary objects that Liu et al. hold as reference points based on results of other researchers\u2019 co-word analyses. (The comparison disciplines used in Liu et al. are psychology, consumer behaviour, software engineering and stem cell research.) These reference points can then be used to show the absence of \u201cQuadrant I\u201d keyword clusters in HCI compared to other disciplines and thus the disciplinary deficiencies of HCI.<\/p>\n<figure><img decoding=\"async\" src=\"https:\/\/66.media.tumblr.com\/de3312bbf4b68f8966c31f900938dcbd\/tumblr_inline_nltvadDNKn1qe6gmj_500.png\" alt=\"image\" \/><\/figure>\n<p>Even if we take HCI as a discipline, the corresponding implication of Liu et al. that disciplines are somehow \u2018comparable\u2019 is itself contentious, I would argue. For instance, it is hard to see how, say, the activities of stem cell researchers have any bearing on the activities of HCI researchers, and it is not clear whether it is reasonable to assume their paper-writing practices, let alone their everyday research work practices, are similar. Or, perhaps, those of psychologists and software engineers. Instead I would suggest that each works with phenomena particular to them, and have methods of reasoning and research practices particular to them. What <i>counts<\/i>\u00a0as relevant research questions in one has nothing necessarily to do with what counts in another. Further, it is also unclear with this disciplinary assumption in place why it might be that specialisms like stem cell research should be compared with <i>all<\/i>\u00a0of psychology\u2014a broad church to say the least\u2014why not social psychology or cognitive psychology? Instead, co-word analysis may be just analysing how keywords (of whatever extraction method) get used, and it may be just that HCI\u2019s use of keywords is <i>different<\/i>\u00a0rather than <i>deficient<\/i>.<\/p>\n<p>The very idea that HCI is a discipline at all is also itself certainly contentious. I think <a href=\"http:\/\/www.morganclaypool.com\/doi\/abs\/10.2200\/S00418ED1V01Y201205HCI014\">Yvonne Rogers is correct when she suggests that HCI is an \u201cinterdiscipline\u201d<\/a>. The implication (intended or not by her use of this term, I don\u2019t know) is that in being an interdiscipline, HCI should indeed have \u201cThe Big Hole\u201d Kostakos identifies, because the very nature of an interdiscipline would be an absence of a disciplinary core. If there were some essential disciplinary core to HCI it would struggle in its role as broker between disciplines (as pointed out by Alan Blackwell recently (<a href=\"http:\/\/www.cl.cam.ac.uk\/~afb21\/publications\">Blackwell, 2015<\/a>)). Even the earliest moments of HCI commenced as a meeting place between cognitive psychologists, software engineers and, to some extent, designers. In other words \u2018we have never have been disciplinary\u2019. <\/p>\n<p>At its most basic the notion of a discipline is an attempt at finding a way of ordering knowledge (<a href=\"https:\/\/www.msu.edu\/~orourk51\/860-Phil\/Handouts\/Readings\/Weingart-ShortHistory-OUP_HoI-2010.pdf\">Weingart, 2010<\/a>). It is not a \u2018natural fact\u2019 and we cannot treat \u2018the discipline\u2019 as transcendent features of a \u2018hidden order\u2019. \u2018A discipline\u2019 is (I\u2019d argue) an epiphenomenon of the particular community of research practice. And it\u2019s precisely because of this that the arguments made about the application of concepts borrowed from (broad brush) \u2018science\u2019 become difficult to handle. Onto which topic I turn next.<\/p>\n<p><\/p>\n<h2>Accumulation, replication, generalisation: On \u2018science\u2019 and \u2018the scientific\u2019 in HCI<\/h2>\n<p>One of the key assertions in the <i>interactions<\/i>\u00a0article is that \u201ca lack of motor themes should be a very worrying prospect for a scientific community\u201d. Kostakos suggests that remedies should be pursued \u201c[if] we want to claim that CHI is a scientific conference\u201d. I interpret this to mean that HCI has the potential for a <i>scientific disciplinarity<\/i>\u00a0that may be established through the development of motor themes. Accordingly, a set of signature scientific procedures or \u2018scientific qualities\u2019, as I\u2019ll label them, are described by Kostakos so as to achieve this; these are mentioned as 1. accumulation (science\u2019s work is that of cumulative progress), 2. replication (science\u2019s work gains rigour from replicability), and 3. generalisation (science\u2019s cumulative work involves expansivity). Kostakos describes how \u201cnew initiatives have sprung up our field to make it more scientific in the sense of repeating studies, incremental research, and reusable findings\u201d, which I take as reference to the replication (Wilson et al., 2011) and \u201cinteraction science\u201d (Howes et al., 2014) agendas I describe above.<\/p>\n<p>Yet making HCI \u201cmore scientific\u201d is not really a new drive in HCI. HCI\u2019s initial development was oriented strongly by many self-described scientists (going by Liu et al.\u2019s scheme of labelling sciences) from psychology and cognitive science, both of which have often been at pains to demonstrate their scientific credentials through adherence methods presumed to be drawn from the natural sciences. So one could argue that the cultural foundations for HCI\u2019s desire to be \u2018scientific\u2019 have always been present. In addition, attempts to reorder HCI back into accord with the \u2018scientific qualities\u2019 outlined by Kostakos have also been suggested before, such as notions from <a href=\"https:\/\/users.soe.ucsc.edu\/~maw\/papers\/envelope-5-15-00-FINAL.pdf\">Whittaker et al. (2000)<\/a><br \/> to develop standardised \u201creference tasks\u201d in order to establish generalisation, and therefore \u2018scientific\u2019 legitimacy. These attempts have faltered, however.<\/p>\n<p>The problem, I think, is firstly that can be very problematic to engage in deployments of \u2018science\u2019 as a concept. Secondly I think it is mistaken at least to imply (or not guard against an implication even if unintended) that these qualities are properties <i>of<\/i>\u00a0\u2018science itself\u2019. <\/p>\n<p>On the first point, \u2018science\u2019 is a linguistic chimera for HCI because the term is so diversely and nebulously applied, not only in Liu et al. and by Kostakos, but also in discourse within HCI more broadly. It is unhelpful for us because \u2018science\u2019 is often used to do <i>very<\/i>\u00a0different things that we may well wish to avoid. For instance, this may be in establishing a kind of epistemic and \/ or moral authority, or an attempt to gain peer esteem for a research community in poor academic standing, or internally as a method for legitimising certain kinds of work and delegitimising others\u2019 (i.e., categorisation between \u2018science\u2019 and \u2018not science\u2019) in the course of cultural wars. \u2018Science\u2019 then becomes problematic because such (rhetorical) uses can tend to be deployed <i>in place of<\/i>\u00a0adequate assessments of research rigour on its own terms (what \u2018own terms\u2019 might mean is explored below).<\/p>\n<p>This leads to my second point, the idea of the accumulation, replication and generalisation of findings as being <i>intrinsic properties of<\/i>\u00a0\u2018science\u2019 rather than methodical practices conducted <i>by<\/i>\u00a0a community of researchers (see <a href=\"http:\/\/www.cs.nott.ac.uk\/~axc\/work\/ECSCW_2013.pdf\">Crabtree et al. (2013)<\/a> and also <a href=\"http:\/\/www.johnrooksby.org\/papers\/CSCW2014_rooksby_replicated.pdf\">Rooksby (2014)<\/a> on this point). This latter view suggests that the standards of \u2018what counts\u2019 as a generalisation, \u2018what is\u2019 a relevant process of accumulation (which I take as the establishing within researchers\u2019 discourse of particular motor themes), and \u2018what motivates\u2019 the conduct of replications, should be decided upon as a matter of agreement between researchers. It cannot be determined through adherence to an external and nebulous set of \u2018scientific standards\u2019 that are adopted from a notion of \u2018science in general\u2019 (e.g., what we might call \u2018textbook\u2019 understandings developed from formal descriptions of the natural sciences)\u2014for no such thing really exists. Instead, if by \u2018science\u2019 we mean \u2018demonstrating a rigour agreed upon by practitioners of the relevant and particular genre of reasoning the work pertains to\u2019 then I might consider it a useful term. But it seems unlikely this is what is being meant (it\u2019s definitely very unwieldy!).<\/p>\n<p>This all said, I have a great deal of sympathy for the desire of Liu et al., Kostakos, Wilson et al., Howes et al., and others who seek to increase the rigour of the HCI community\u2014such a motivation for the critique <i>can only be encouraged<\/i>. Yet, to reiterate, this cannot come at the expense of specifying singular-yet-nebulous approaches like making HCI \u2018more scientific\u2019, particularly when the model of \u2018more scientific\u2019 is based on classic tropes of what a mythical \u2018science\u2019 is said to be, rather than as a matter of how researchers engage in the various shared practices to establish agreement and disagreement over findings. <\/p>\n<p>Instead, I think if we take the \u2018interdiscipline\u2019 challenge seriously we should be looking for two things of particular HCI contributions. Firstly, we should expect a rigour commensurate with the research\u2019s own disciplinary wellsprings, whether this is (cognitive, social, etc.) psychology, anthropology, software engineering or, more recently, the designerly disciplines. Rare examples of such \u2018internal rigour\u2019 being taken to task is found in the \u2018damaged merchandise\u2019 (<a href=\"http:\/\/homepages.rpi.edu\/~grayw\/pubs\/papers\/1998\/Gray&amp;Salzman98a_HCI.pdf\">Gray and Salzman, 1998<\/a>), \u2018usability evaluation considered harmful\u2019 (<a href=\"http:\/\/www.billbuxton.com\/usabilityHarmful.pdf\">Greenberg and Buxton, 2008<\/a>) or \u2018ethnography considered harmful\u2019 (<a href=\"http:\/\/www.cs.nott.ac.uk\/~axc\/work\/CHI09.pdf\">Crabtree et al., 2009<\/a>) debates (although in HCI they feel like more like \u2018scandals\u2019\u2014which perhaps says something about the level of debate in HCI more than anything else). What this means is that the adoption of materials, approaches, perspectives, etc. from disciplines \u2018external\u2019 to HCI (and remember in this view, there is only \u2018the external\u2019) should not result in lax implementations of such imported concepts, approaches, etc. within the HCI community. The \u2018magpie-ism\u2019 of HCI research is a double-edged sword: increasing vigour and research creativity, yet often resulting in violence being done to the origins of imported approaches, concepts, etc. And without specialist attention, weak strains are sustained \/ incubated within HCI; the controversies outlined above are manifestations of this problem. Secondly, we should expect a rigour in the HCI research contribution\u2019s engagement with the notion of <i>being<\/i>\u00a0an \u2018interdiscipline\u2019. This is what \u2018implications for design\u2019 is all about (albeit quite a deficient form as pointed out by Kostakos and others); that is, an attempt to <i>meet others at the interface of disciplines<\/i>. But more on this next.<\/p>\n<p><\/p>\n<h2>The curse of the interdiscipline: Implications for design<\/h2>\n<p>The <i>interactions<\/i> article builds upon Liu et al. by arguing that \u201cthe reason our discipline lacks mainstream themes, overarching or competing theories, and accumulated knowledge is the culprit known as implications for design\u201d. The absence of HCI\u2019s engagement with proper \u2018scientific qualities\u2019 like generalisation and accumulation is thus pinned to the perceived need to write \u201cimplications for design\u201d sections in CHI papers in order to get them past peer review even when the rest of a paper is presenting a high quality of research work. Moreover, within the interdisciplinary community of HCI, it really can never be enough, as Kostakos rightly points out, just to vaguely target \u2018relevant practitioners\u2019. <\/p>\n<p>While I have sympathy for this argument, I also think a reassessment has to be made as to <i>why<\/i>\u00a0the \u2018implications for design\u2019 discussion has emerged in the first place, which I have hinted at above. We can use similar questions to those posed over keywords: what is \u2018being done\u2019 in the writing of \u2018implications for design\u2019? (Helpfully, Sas et al. (2014) have recently published a categorisation of the different kinds of uses \u2018implications of design\u2019 is put to.)<\/p>\n<p>I would argue that \u2018implications for design\u2019 can be read as a gesture towards being an \u2018interdiscipline\u2019. They are typically an effort to answer the question \u201cwhy should I (the reader) care about this work?\u201d, a question that is in no way unique to HCI. It would be a mistake to assume that we need not be accountable to the \u2018interdisciplinary other\u2019 in HCI. And yes, often the gesture is poorly performed and poorly labelled. <\/p>\n<p>Instead we should perhaps start considering \u2018implications for HCI\u2019 rather than \u2018implications for design\u2019 as a better sign of taking work at the interface of disciplines seriously.<\/p>\n<p><\/p>\n<h2>Update (22\/04\/15)<\/h2>\n<p><a href=\"http:\/\/transground.blogspot.kr\/2014\/05\/a-growing-problem-in-hci-research.html\">Erik Stolterman has expressed similar concerns<\/a> about HCI\u2019s\u00a0\u2018core\u2019, see his blog post.<\/p>\n<p><a href=\"https:\/\/interactionculture.wordpress.com\/2014\/05\/26\/hci-as-core-or-relation\/\">Jeff Bardzell has responded to this in a blog post<\/a>, arguing that it may be better to conceptualise HCI as a set of relations (i.e., it has a relational identity) rather than having a core.<\/p>\n<p><\/p>\n<h2>References<\/h2>\n<p>Blackwell, A. F. (2015). <a href=\"http:\/\/www.cl.cam.ac.uk\/~afb21\/publications\/\">HCI as an inter-discipline<\/a>. To appear in <i>Proc. CHI 2015 (alt.chi)<\/i>.<\/p>\n<p>Carroll, J. M. and Campbell, R. L. (1986). <a href=\"http:\/\/citeseerx.ist.psu.edu\/viewdoc\/download?doi=10.1.1.22.4849&amp;rep=rep1&amp;type=pdf\">Softening up Hard Science: reply to Newell and Card. Human\u2013Computer Interaction<\/a>, 2(3):227-249, Taylor and Francis, 1986.<\/p>\n<p>Crabtree, A., Rodden, T., Tolmie, P., and Button, G. (2009). <a href=\"http:\/\/www.cs.nott.ac.uk\/~axc\/work\/CHI09.pdf\">Ethnography considered harmful<\/a>. In <i>Proc. CHI 2009<\/i>.<\/p>\n<p>Crabtree, A., Tolmie, P. and Rouncefield, M. (2013). \u2018<a href=\"http:\/\/www.cs.nott.ac.uk\/~axc\/work\/ECSCW_2013.pdf\">How many bloody examples do you want?\u2019 &#8211; fieldwork and generalisation<\/a>. In Proc ECSCW 2013.<\/p>\n<p>Gray, W. D. and Salzman, M. C. (1998). <a href=\"http:\/\/homepages.rpi.edu\/~grayw\/pubs\/papers\/1998\/Gray&amp;Salzman98a_HCI.pdf\">Damaged merchandise? a review of experiments that compare usability evaluation methods<\/a>. <i>Hum.-Comput. Interact.<\/i>, 13, 3 (September 1998), 203-261.<\/p>\n<p>Greenberg, S. and Buxton, W. (2008). <a href=\"http:\/\/www.billbuxton.com\/usabilityHarmful.pdf\">Usability evaluation considered harmful (some of the time)<\/a>. In<i> Proc.\u00a0CHI 2008<\/i>.\u00a0<\/p>\n<p>Howes, A., Cowan, B. R., Payne, S. J., Cairns, P., Janssen, C. P., Cox, A. L., Hornof, A. J., and Pirolli, P. (2014). <a href=\"http:\/\/chi2014.acm.org\/spotlights\/CHI-Science-proposal-final.pdf\">Interaction\u00a0Science Spotlight<\/a>.<i>\u00a0CHI 2014<\/i>.<\/p>\n<p>Kostakos, V. (2015). <a href=\"http:\/\/www.ee.oulu.fi\/~vassilis\/files\/papers\/interactions15.pdf\">The big hole in HCI research<\/a>. <i>interactions<\/i> 22, 2 (February 2015), pp. 48-51. <\/p>\n<p>Liu, Y., Goncalves, J., Ferreira, D., Xiao, B., et al. (2014). <a href=\"http:\/\/www.ee.oulu.fi\/~vassilis\/files\/papers\/chi14.pdf\">CHI 1994\u20132013: Mapping two decades of intellectual progress through co-word analysis<\/a>. In <i>Proc.\u00a0CHI 2014<\/i>.<\/p>\n<p>Newell, A. and Card, S. K. (1985). <a href=\"http:\/\/www.ics.uci.edu\/~redmiles\/inf233-FQ07\/oldpapers\/Prospects.pdf\">The prospects for psychological science in human-computer interaction<\/a>. <i>Hum.-Comput. Interact<\/i>. 1, 3 (September 1985), pp. 209-242.\u00a0<\/p>\n<p>Rogers, Y. (2012). <i><a href=\"http:\/\/www.morganclaypool.com\/doi\/abs\/10.2200\/S00418ED1V01Y201205HCI014\">HCI Theory: Classical, Modern, and Contemporary<\/a><\/i>. Morgan &amp; Claypool, May 2012.<\/p>\n<p>Rooksby, J. (2014).\u00a0<a href=\"http:\/\/www.johnrooksby.org\/papers\/CSCW2014_rooksby_replicated.pdf\">Can Plans and Situated Actions Be Replicated? <\/a>In <i>Proc. CSCW 2014.<\/i><\/p>\n<p>Sas, C., Whittaker, S., Dow, S., Forlizzi, J., and Zimmerman, J. (2014). <a href=\"http:\/\/www.cs.cmu.edu\/~spdow\/files\/implicationsDesignResearch-CHI2014.pdf\">Generating implications for design through design research<\/a>. In <i>Proc.\u00a0CHI 2014.<\/i><\/p>\n<p>Weingart, P. (2010). <a href=\"https:\/\/www.msu.edu\/~orourk51\/860-Phil\/Handouts\/Readings\/Weingart-ShortHistory-OUP_HoI-2010.pdf\">A short history of knowledge formations<\/a>. In Thompson, J. Klein and Mitcham, C. (eds.), <i>The Oxford Handbook of Interdisciplinarity<\/i>. OUP Oxford, pp. 3-14.<\/p>\n<p>Whittaker, S., Terveen, L., and Nardi, B. A. (2000). <a href=\"https:\/\/users.soe.ucsc.edu\/~maw\/papers\/envelope-5-15-00-FINAL.pdf\">Let\u2019s stop pushing the envelope and start addressing it: a reference task agenda for HCI<\/a>. <i>Hum.-Comput. Interact<\/i>. 15, 2 (September 2000), pp. 75-106.<\/p>\n<p>Wilson, M. L., Mackay, W. E., Chi, E. H., Bernstein, M. S., Russell, D., Thimbleby, H. W. (2011). <a href=\"http:\/\/www.cs.nott.ac.uk\/~mlw\/pubs\/RepliCHI-panel_CR.pdf\">RepliCHI\u2014CHI should be replicating and validating results more: discuss<\/a>. <i>CHI Extended Abstracts 2011<\/i>: pp. 463-466.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"<p>notesonresearch: At CHI 2014 a paper was presented (Liu et al., 2014) which sought to demonstrate, through an analysis of keywords specified in a large tranche of CHI papers (from the last 20 years), that HCI research is lacking in \u201cmotor themes\u201d. According to the paper, a motor theme is a commonly addressed topic in [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[10],"tags":[],"class_list":["post-12824","post","type-post","status-publish","format-standard","hentry","category-words"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p6PWot-3kQ","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/rafaelfajardo.com\/portfolio\/wp-json\/wp\/v2\/posts\/12824","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rafaelfajardo.com\/portfolio\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rafaelfajardo.com\/portfolio\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rafaelfajardo.com\/portfolio\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rafaelfajardo.com\/portfolio\/wp-json\/wp\/v2\/comments?post=12824"}],"version-history":[{"count":0,"href":"https:\/\/rafaelfajardo.com\/portfolio\/wp-json\/wp\/v2\/posts\/12824\/revisions"}],"wp:attachment":[{"href":"https:\/\/rafaelfajardo.com\/portfolio\/wp-json\/wp\/v2\/media?parent=12824"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rafaelfajardo.com\/portfolio\/wp-json\/wp\/v2\/categories?post=12824"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rafaelfajardo.com\/portfolio\/wp-json\/wp\/v2\/tags?post=12824"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}