Parallel Papers
Reading non-verbal expressions of anger and embarrassment
Christine Anthonissen & Marcelyn Oostendorp, Stellenbosch University
This paper is interested in the semiotic means of expressing and interpreting anger and embarrassment in oral discourse where the speaker is in a relatively weak position of power. If speakers of the same language and from similar socio-cultural background use (e.g.) kinesics and voice quality to give indications of emotional responses during conversation, there is no guarantee that there will be perfect understanding between participants. If the speakers have different first languages and different socio-cultural circumstances, the likelihood of misunderstanding expressed emotions, is considerably higher. Even mediation by the intervention of an interpreter, appears to be a precarious exercise in the making of meaning in such a context. Working with data collected at an HIV/AIDS day clinic where doctors and patients do not share the same linguistic repertoires, we shall investigate some of the theoretical means that may assist
(i) in recognizing misunderstanding regarding emotions expressed non-verbally,
(ii) in explaining the misunderstanding regarding emotions expressed non-verbally, and
(iii) in considering discursive possibilities of repair in the given circumstances.
The paper will specifically refer to theoretical bases of Critical Discourse Analysis and to work by Ana Pavlenko on Emotions and Multilingualism in considering how participants process non-verbal communicative signifiers in emotionally intense conversation.
Hjelmslev, SFL and Semiotic Margins
Carl Bache, Institute of Language & Communication,
In this paper I want to do three things. First I will offer a review of Louis Hjelmslev and his formal approach to language known as Glossematics, paying special attention to his Omkring Sprogteoriens Grundlæggelse from 1943 (= Prolegomena to a Theory of Language, translated by Francis J. Whitfield in 1961). Secondly I’ll try to place Hjelmslev’s contribution to linguistics in a wider structuralist context (and especially trace the elements that may have served as a source of inspiration for SFL). Finally I will discuss the possible usefulness of linguistic immanence as a contrastive starting point for an analysis of semiotic margins. Time permitting, I will show how some of Hjelmslev’s ideas may be developed into a new, more radical approach to paradigmatic relationships
Meaning rough: sound quality and intensification in North American Rap Music
David Caldwell, University of Sydney
‘Rapping’ as a form of musical expression is often characterised according to an artists’ use of rhythm and voice quality. Krims (2000) for example has defined three genres of North American rap music, with rhythm and timbre as his main parameters.
For this paper, I will also investigate the rapping style or ‘flow’ of several mainstream North American rap artists. Following the Systemic Functional Linguistic (SFL) tradition, I will draw on van Leeuwen’s (1999) system networks for time and sound quality to analyse the data. From a social semiotic perspective, I expect to assign interpersonal meaning potentials to these rhythms and sound qualities. In particular, I will follow the APPRAISAL system network of GRADUATION (Martin and White 2005) to identify the extent to which rhythm and sound quality ‘intensifies’ the ATTITUDE expressed in the verbiage. Moreover, I expect to use that initial analysis to describe periodicity in rap songs; the ways in which the rhythm and voice quality unfold into waves of ‘intensification’.
Ultimately, this paper seeks to contribute to the small body of research (e.g. Callaghan and McDonald 2002) that has begun to theorise a SFL model of music. It is hoped that investigating the ‘human voice’ as both a mode of linguistic expression as well as musical expression may help to blur our semiotic margins.
Callaghan, J. and McDonald, E. 2002. Expression, Content and Meaning in Language and Music: an integrated semiotic approach. In McKevitt, P. Ó’Nualláin, S. and Mulvihill, C. (eds). Language, Vision, Music. Amsterdam: Benjamins, pp. 205-230.
Krims, A. 2000. Rap Music and the Poetics of Identity. Cambridge: Cambridge University Press.
Martin, J.R. and White, P.R.R. 2005: The Language of Evaluation: Appraisal in English. New York: Palgrave Macmillan.
van Leeuwen, T. 1999: Speech, Music, Sound. London: Macmillan.
Integrating visual and verbal meaning in multimodal text comprehension: towards a model of intermodal relations
Eveline Chan, UNE
In contexts for literacy teaching and learning, it is now commonplace to recognise the contribution of non-language modalities in the construction of meaning. However, in the assessment of students’ comprehension of multimodal texts, constructs of reading literacy are still strongly oriented towards the written mode. One reason for this mismatch in practice is the lack of a substantive theoretical framework that enables test writers to address how images and print, both separately and in combination, contribute to overall text meaning.
This paper presents some preliminary findings from an ARC Linkage project on the visual and verbal dimensions of reading comprehension in school literacy tests. One of the goals of the project is to develop a model of intermodal relations which will reveal the extent to which visual and verbal elements contribute to the overall meaning in multimodal texts, and the nature of these relationships among the elements. Drawing on SFL-derived approaches to multimodal text analysis, the model uses Halliday’s metafunctional framework as an organising principle, and takes the interplay of representational and compositional meaning as its starting point. The model is initially being applied to NSW school literacy test materials and will be refined in response to feedback from subsequent test cycles as more data is generated. The results to date suggest that comprehension of more complex multimodal texts is often dependent upon recovering meaning from different types of relationships that extend across the modalities.
Persuasive strategies employed by television advertisments: an analysis of consumer experiences created in Bacardi Breezer ads using SF-MDA
Chan Yoke Hian, National University of Singapore
The Systemics-Functional (SF) approach first introduced into Multimodal Discourse Analysis (henceforth SF-MDA) by O’Toole (1994) has been developed and adapted for film by O’Halloran (2004) , Internet websites by Baldry & Thibault (2006) and Djonov (2006) and television advertisements by Thibault (2000) and Tan (2005) amidst other researchers in the fields of Internet studies, education, linguistics and semiotics. There is general consensus that the multimodal items interact across levels to create meanings. However, the analysis of video texts remains a challenge for SF-MDA analysis (Djonov, 2006; Jones, 2007) due in part to the dynamic nature of these texts and the difficulty of trying to capture the meanings and their interactions across different semiotic modes.
Through an analysis of Bacardi Breezer television advertisements, I attempt to widen the scope of previous multimodal analyses in three ways.
Firstly, in contributing to improving multimodal transcription methodologies previously
employed by Thibault (2000), O’ Halloran (2004), Tan (2005), I will show how the inclusion of the visual representation of the acoustic flux into previous templates can contribute valuable insights to our understanding of compositional, interpersonal and representational transitions within TV ads.
Secondly, I illustrate how the notion of phasal triggers can be useful for tracing the
reconstitution and contextualisation of these dynamic texts via semiotic metaphors
(O’Halloran, 2005). The shifts occurring between and within phases-shots-frames (Thibault, 2000) and the transitions from processes-qualities-entities (Halliday, 1994 and Halliday & Matthiessen 2004) within the Bacardi Breezer ads motivate a re-configuration of our conventional representation of rank-scale relations (O’Toole, 1994).
Thirdly, I introduce how the interaction of different semiotic modes through stereotypic
enactment, reconstruction and juxtaposition create meanings via contrastive salience in these advertisements which are used to shape consumer experiences.
Finally, I attempt to frame the analysis using a Bourdieu perspective (1991) in order to situate the analysis to the broader context of contemporary cultural theory. Using notions of the habitus, field and capital, I illustrate how the consumer experiences created through the advertisements project these ads as non-conventional and subversive, which actually contribute to their normalising and re-affirming other ideological notions.
Baldry, A. and Thibault, P (2006) Multimodal Transcription and Text Analysis. London:
Equinox.
Bourdieu, P. (1991) Language and Symbolic Power. Cambridge, Harvard University Press.
Djonov, E. (2006) Analysing the organisation of information in websites: From Hypermedia Design to Systemic Functional Hypermedia Discourse Analysis. PhD. Thesis, School of English and School of Modern Language Studies, University of New South Wales.
Halliday, M. A. K. (1994) An Introduction to Functional Grammar. 2nd edition. London: Arnold.
Halliday, M. A. K. and Matthiessen, C. (2004) An Introduction to Functional Grammar. 3rd edition. London: Arnold.
Jones, J. (2007) Multiliteracies for Academic Purpose: A Metafunction Exploration of
Intersemiosis and Multimodality in University Textbook and Computer-based Learning
Resources in Science. EdD. Thesis, University of Sydney.
O’Halloran, K. (2004) “Visual Semiosis in Film”. In Kay O’ Halloran (ed.), Multimodal
Discourse Analysis: Systemic Functional perspectives. London and New York: Continuum, pp. 109-113.
O’Halloran, K. (2005) Mathematical Discourse: Language, Symbolism and Visual Images. London and New York: Continuum.
O’Toole, M. (1994) The Language of Displayed Art. London. Leicester University Press.
Tan, S. (2005) A Systemic Functional Approach to the Analysis of Corporate Television
Advertisements. MA thesis, Department of English Language and Literature, National
University of Singapore.
Thibault, P. (2000) “The Multimodal Transcription of a Television Advertisement: Theory and Practice”, in Anthony Baldry (ed.) Multimodality and Multimediality in the Distance Learning Age. Campobasso, Italy: Palladino Editore, pp. 331-385.
Towards a relational grammar of image-verbiage synergy: intermodal representations
Chris Cleirigh, University of Sydney/UNE
Len Unsworth, UNE
Researchers within systemic-functional semiotics have sought to theorize image-verbiage relations from an experiential perspective using an adapation of Halliday and Hasan’s (Halliday & Hasan, 1976) lexical cohesive relations to analyse magazine articles (Royce, 1998) and school science textbooks (Royce, 2002), and by proposing a notion of semiotic metaphor in relation to mathematics texts (O'Halloran, 1999, 2003a, 2003b). Recently Martinec and Salway (2005) have proposed a system of image-text relations derived from a combination of Halliday’s (1994) logico-semantic and status relations developed for classifying clauses in the clause complex, and Barthes’ (1997 (1961); Barthes, 1997 (1964)) text relations dealing with photographs in newspapers and to a lesser extent moving images and dialogue in film texts.
This session firstly notes a number of inconsistencies and contradictions in the Martinec and Salway analyses and then explores an alternative approach to modelling image-verbiage relations derived from the Token/Value relations of identifying clauses. Intensive, possessive and circumstantial identifying relations can be used to model the ways in which verbiage glosses images and images simultaneously visualize verbiage. Sample analyses using primary school science materials will be provided followed by a re-analysis of the Martinec and Salway data using the relational grammar of image-verbiage synergy proposed here. Potential application of this approach to other types of texts such as picture books will be briefly noted, as will be implications for multimodal literacy pedagogy.
Barthes, R. (1997 (1961)). The photographic message. In R. Barthes (Ed.), Image-Music-Text (pp. 15-31). London: Fontana.
Barthes, R. (1997 (1964)). Rhetoric of the image. In R. Barthes (Ed.), Image-Music-Text (pp. 32-51). London: Fontana.
Halliday, M. A. K. (1994). An introduction to functional grammar (2 ed.). London: Edward Arnold.
Halliday, M. A. K., & Hasan, R. (1976). Cohesion in English. London: Longman.
Martinec, R., & Salway, A. (2005). A system for image-text relations in new (and old) media. Visual Communication, 4(3), 337-371.
O'Halloran, K. (1999). Interdependence, Interaction and Metaphor in Multisemiotic Texts. Social Semiotics, 9(3), 317-338.
O'Halloran, K. (2003a). Implications of mathematics as a multisemiotic discourse. In M. Anderson, A. Saenz-Ludlow, S. Zellweger & V. Cifarelli (Eds.), Educational perspectives on mathematics as semiosis: From thinking to interpreting to knowing (pp. 185-214). Brooklyn/Ottawa/Toronto: Legas Publishing.
O'Halloran, K. (2003b). Intersemiosis in mathematics and science: Grammatical metaphor and semiotic metaphor. In A.-M. Simon-Vandenbergen, M. Taverniers & L. Ravelli (Eds.), Grammatical Metaphor (pp. 337-366). Amsterdam/Philadelphia: John Benjamins.
Royce, T. (1998). Synergy on the page: Exploring intersemiotic complementarity in page-based multimodal text. Japan Association Systemic Functional Linguistics Occasional Papers, 1(1), 25-50.
Royce, T. (2002). Multimodality in the TESOL classroom: Exploring visual-verbal synergy. TESOL Quarterly, 36(2), 191-205.
(The Salient Marginality of) Sound and Movement as Paravisual Resources in Children’s Websites
Emilia Djonov, Macquarie University
Usability prescriptions against the use of movement and sound in websites have played a limited role in the design of websites for children. Until recently, perhaps in an attempt to attract and support children who cannot yet read in conventional terms, these semiotic modes were employed predominantly to increase the salience of or merely repeat visually and/or verbally realised meanings (e.g. when the inclusion of section icons in a given visual composition and their behaviour as rollovers both serve to present a website’s sections as similar to each other or when the instructions for playing a game are spoken as well as written). In the past 3-4 years, however, children’s webpage design increasingly displays what Halliday (1991) refers to as the “disassociation of associated variables” – a semogenic process whereby resources of expression (e.g. movement, colour and sound) become unhinged from other resources they are typically co-deployed with as well as from the meanings they are typically used to realise (independently or in collaboration with other resources).
This paper will explore this process in the webpage design of two popular edutainment websites for children National Geographic Kids (National Geographic Society, 1996-2007) and The Playground (Australian Broadcasting Corporation, 1998-2007). In particular, it extends existing systemic-functional accounts of the use of dynamic resources in websites (cf. Baldry & Thibault, 2006) by considering the extent to which kinetic and audio resources depend on the integrative visual realisation of multimodal principles such as framing, salience, layout and composition (Kress & van Leeuwen, 2001, 2006 [1996]) for the realisation of ideational, interpersonal and textual meanings in different versions of each website. This exploration will show that although kinetic and audio resources still depend on visual ones, without these dynamic resources webpage design is limited in its potential to reflect website fluidity and variety by allowing users to preview website content. Describing sound and movement in webpage design as ‘paravisual’ draws attention to their meaning-making potential both ‘beside’ and ‘beyond’ those functions that visual resources can fulfil on their own (cf. Nöth, 1990 on paralanguage). The paper concludes by discussing why the potential of sound and movement to establish continuity across webpages remains relatively unexploited.
References
Australian Broadcasting Corporation. (1998-2007). The Playground. Retrieved 8 September 2007, from http://www.abc.net.au/children
Baldry, A., & Thibault, P. (2006). Multimodal Transcription and Text Analysis. London: Equinox.
Halliday, M. A. K. (1991). Corpus studies and probabilistic grammar. In K. Aijmer & B. Altenberg (Eds.), English Corpus Linguistics: Studies in Honour of Jan Svartvik (pp. 30-43). London: Longman.
Kress, G., & van Leeuwen, T. (2001). Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold.
Kress, G., & van Leeuwen, T. (2006 [1996]). Reading Images: The Grammar of Visual Design. London: Routledge.
National Geographic Society. (1996-2007). National Geographic Kids. Retrieved 8 September, 2007, from http://www.nationalgeographic.com/kids
Nöth, W. (1990). Handbook of Semiotics. Bloomington/ Indianopolis: Indiana University Press.
Grappling with a non-speech language: describing and theorizing the nonverbal multimodal communication of a child with an intellectual disability
Shooshi Dreyfus, Wollongong University
Based on a case study of the nonverbal multimodal communication of a child with an intellectual disability, this paper explores the modes of communication the child uses instead of speech, such as non-speech vocalizations, gestures, actions and behaviours, and also attempts to grapple with how one theorises such nonverbal modes of communication.
In a world where all behaviour is viewed as communication, “paralinguistic” behaviour needs to be seen as central rather than relegated to a support role for speech. However, given that there is no lexicogrammatical stratum, it becomes more difficult, both for communication partners and for the analyst, to deal with such non-verbal meanings. This paper discusses how the study both described and analysed a range of modes of communication, attempting to theorise about the variety of modes of behaviour, using the resources of systemic functional theory.
Visual graduation: flagging attitude in broadsheet news photos
Dorothea Economou (Ph.D candidate Sydney University)
Abstract :
This paper reports on a study of ‘factual' broadsheet news photos – those captioned naturalistic colour photos appearing with daily news stories in which ‘congruent' visual ideational meanings predominate. It is proposed here that in this factual visual genre, just as in some verbal genres where inscribed attitude is also highly constrained, an evaluative reading is systematically flagged by the graduation of ideational meanings. This paper will present the application and adaptation to news photos of the Graduation network as developed for linguistic text in Appraisal theory (Martin & White, 2005; Hood, 2004a,b). It will be demonstrated that the interpersonal impact (force) and interpersonal value (focus) of visual meaning can be raised or lowered by simultaneous selections from the range of gradient ‘spatial' and ‘textural' expression systems available to naturalistic photography.
References :
Hood, S. 2004a Appraising Research: Taking a stance in academic writing, PhD Faculty of Education, UTS
Hood, S. 2004b Managing attitude in undergraduate writing In Ravelli, L. & Ellis, R. (eds) Analysing academic writing: contextualised framework . London: Continuum, 24-44
Martin, J.R. & White, P.R.R. 2006 The language of evaluation: Appraisal in English London : Palgrave Macmillan
Iconic and Proud: Aspect in Australian Signed Language (Auslan)
Michael Gray, University of Sydney
Signed languages have offered an interesting challenge to many linguistic frameworks, particularly in the high degree of perceived iconicity in their lexicons and grammars. This presentation will focus on the iconicity of aspect marking on verbs in Auslan to show that signed languages are iconic and (should be) proud.
Beginning from a pilot study using data from Dr Trevor Johnston's Auslan corpus, we will examine the manner in which aspectual information can be conveyed on verbs in Auslan. By examining verbs modified to show aspectual information in the context of the narratives where they occur, and comparing this language to the ‘actual' event, it will be argued that Auslan makes use of highly visually iconic grammatical processes, and that these processes do not fit the criteria of ‘inflectional morphology' to which they are sometimes assigned.
Far from a problem for the recognition of Auslan as a language, this analysis of aspect marking is congruent with a growing recognition of the place of gestural and visual representation in the world's signed languages. Since these forms of representation are often marginalised in the study of human language when this study focuses on spoken languages, this presentation suggests a role for signed languages in encouraging a more inclusive appreciation of what constitutes human language.
Literary typography: mise-en-page in advertising and poetry
Rosemary Huisman, University of Sydney
Contemporary print advertising and contemporary printed poetry seem to have developed a symbiotic relationship, in which each appears to feed off the typographic conventions and innovations of the other. In this paper, I look particularly at the mise-en-page, or layout, of examples of each, and consider the possible functions/purposes (in systemics, meaning) of similar practices in the different discourses.
Colour in learning resources for science students at university: pedagogical and design motivations
Janet Jones, University of Sydney
Recent literature on multimodality has discussed the non-marginal status of colour as a semiotic resource (Kress & van Leeuwen, 1996, 2002; Lim 2004; Thibault, 2000). Within this discussion, researchers continue to debate the extent to which colour can function as a semiotic system in its own right, with its own grammar. There is general agreement however, that colour works metafunctionally, simultaneously realising ideational, interpersonal and textual meanings. Yet colour cannot work independently of other semiotic resources such as image and text, and its use will vary according to context and groups of users. In a pedagogical context, colour schemes have become an important feature of print and screen-based learning resources and are influenced by ‘fashions' in the global practices of publishing and technical design. Colour schemes are built-in and stable features in these learning resources, integral to their graphic and instructional design.
This paper will first explore the use of colour and colour schemes in textbook and online learning resources for university science students and how colour can be seen to contribute in different ways to metafunctional meaning in image and text. The second part of the paper will demonstrate how the resources of colour have been used in the design of an online program which aims to teach university students about the scientific language and genre of a laboratory report.
Kress, G., & van Leeuwen, T. (1996). Reading images: The grammar of visual design. London: Routledge.
Kress, G., & van Leeuwen, T. (2002). Colour as a semiotic mode: Notes for a grammar of colour. Visual Communication, 1(3), pp. 343-368.
Lim, F. V. (2004). Developing an integrative multisemiotic model. In K. O'Halloran (Ed.), Multimodal discourse analysis: Systemic functional perspectives (pp. 220-246). London: Continuum.
Thibault, P. J. (2000). The multimodal transcription of a television advertisement: Theory and practice. In A. P. Baldry (Ed.), Multimodality and multimediality in the distance learning age (pp. 311-385). Campobasso, Italy: Palladino Editore.
Colour Lexis in Visual Grammar
Francis Low, Hong Kong Polytechnic University
Of all forms of non-verbal communication, colour is the most instantaneous method of conveying messages and meanings. Among other uses, colour stimulates and works synergistically with all of the senses, symbolizes abstract concepts and thoughts, expresses fantasy or wish fulfilment, recalls another time and place and produces an aesthetic or emotional response.
Kress and Van Leeuwen (1996) discussed colour modality, colour contextualization, their representation, etc. But before we go into all these applications we might still ask what is colour? How does this colour language work on us? And what is the nature of colour doing in the organization of images? This paper explores the fundamentals of how colour builds its world and what constitute colour 'lexis' in visual grammar. Based on Kress and Van Leeuwen's concepts/ideas I would like to take a step back to look at colour and its components in order to establish a system to read colour in context.
Bringing sound into the picture: methodologies for the multimodal analysis of affect in film
Betty Noad, UNE
Sound is marginalised in semiotic analyses of multimodal texts such as film, despite its position as a significant communicative resource. Researchers consider it problematic to investigate sound semiosis in film, as research emphasis continues to privilege attention to visual image and dialogue, and analytic concepts and tools for analysing sound in film that are emerging are conceptualised differently. Researchers who do include sound in multimodal analysis adopt different ways of segmenting the text to support analysis and descriptions of meaning, for example researchers might use the concept of rank (OToole 1994; OHalloran 2004), or phases (Baldry & Thibault 2006), or phrase (van Leeuwen, 1999). Further, the different ways in which semiotic resources are theorised makes intersemiotic analysis problematic, for example linguistic and visual resources are theorised according to metafunction, whereas sound resources are theorised according to materiality (van Leeuwen, 1999).
This paper brings the marginalised resources of sound clearly into the picture of multimodal analysis, using The Queen filmtrailer (2006) as a model. It elaborates on the analytical methodologies selected to solve the problems of investigating the role of sound in the construction of affective meaning in film. As well it elaborates on methodologies used to analyse the process of intersemiosis (the integration of multiple semiotic resources) that realises new multimodal affective meanings. In particular this paper demonstrates how sound analysis can be the means for segmenting a filmtext so that semiosis in sound, image and language can be investigated. This paper also points out that such an analytical approach aligns with the perspective of film theorists who argue that viewer attention, and viewer understanding of affective meanings, are manipulated by the co-patterning and integration of filmic information, and that sound takes a critical role in the process.
Meaning beyond the margins: learning to interact with books
David Rose, University of Sydney
Relations between spoken and written modalities have been well described by Halliday 1989, Halliday & Martin 1993 and others, in terms of alternative ways of construing experience, and alternative patterns of organising discourse. But a less well explored aspect of intermodality is from the perspective of social interaction. From the interpersonal angle the striking contrast between spoken and written discourse is that speaking typically involves interacting directly with one or more other people, whereas reading and writing involve interacting not with a person but with a book (or its electronic equivalent). How does personal interaction become written interaction? How do children learn to interact with books? These questions will be explored in this paper, drawing on some recent research in learning to read, and strategies for engaging readers in written discourse, as well as designed techniques for teaching children to read.
Silences, Pauses, syllable length and rhythm: on the semiotic margins of speech
Brad Smith, Macquarie University
For Halliday (1967) ‘Whenever we describe a language we are concerned with meaning, and all contrast in meaning can be stated either in grammar or in lexis’ (p. 10); and on this basis he made his description of the intonation of English grammar. Working from the same basis, I examine excerpts from a (relatively) large corpus of spoken data I have transcribed for intonation using the SFL framework, in terms of various interesting sound phenomena which do not fall within the current SFL description. For example, I have found it necessary in the analysis to distinguish between a Pause and a Silence – the former phonologically systemic, being defined as a rhythmic absence of sound, that is, a silent Ictus in connected speech; the latter being an absence of sound with no systemic relation to the text (although potentially significant for a discourse semantic description) - and between these and the Remiss sound ‘gaps’ between one tone group the next. These and other phenomena, such as the use of rhythm and the shifts between different rhythms, and the temporal duration of Remiss syllables, appear to be functionally motivated, describable as part of the textual metafunction. I will present several instances of spoken text as the starting point for a discussion of the semiotic margins of speech, commencing with the systems of RHYTHM and SALIENCE which, I have claimed (following Brazil 1981: 46-47) elsewhere (ASFLA 2007 Conference), realise the grammmatical systems INFORMATION GROUPING and INFORMATION PROMINENCE.
Brazil, D (1981) Intonation, in Coulthard, M., Montgomery, M. (eds) Studies in discourse analysis. London, Boston, Melbourne, Henley: Routledge and Kegan Paul: pp. 39-50.
Halliday, M. A. K. (1967). Intonation and grammar in British English. The Hague; Paris: Mouton.
Co-articulating “worry” in children’s picture books
Ping Tian, University of Sydney
There is little doubt that in our everyday lives we are surrounded by texts that combine multiple modalities. Among these texts, children’s picture books are, in particular, rich in meaning making with the co-articulation of verbal and visual modes. Furthermore, they usually have a rich emotional repertoire in helping and leading young readers to recognize and process a wide range of emotions.
The current paper examines images that are selected from a children’s picture book named “Silly Billy”, which is written and illustrated by an award-winning author-illustrator Anthony Browne. “Silly Billy” tells the story of a boy called Billy, who worries about everything and has sleepless nights until a visit to his grandma's shows him how to overcome his fears with the aid of his own imagination and some tiny worry dolls.
Throughout the story, various facial expressions are employed to represent “Billy’s worries”. The frequent use of “worry” in verbal texts and its various realizations in visual mode are the concerns of the current paper. In order to understand the relationship between verbiage and image, this paper conducts an APPRAISAL analysis on the text to answer the question of how emotions are represented verbally”. Moreover, this paper carries out an APPRAISAL analysis on the images and in particular, the “facial affect” of the depicted characters to understand how emotions are represented visually. Combing the analysis on two different modes, the current study aims at gaining insights into how images and verbiage co-articulate and create evaluative meanings in bi-modal texts.
How voice quality features create meaning potentials and form alignments in telephone conflicts
WAN Yau Ni (Jenny), Hong Kong Polytechnic University
In call centre telephone interactions customer service representatives (CSR) are expected to constantly maintain high levels of interpersonal interaction with customers. The exchange of interpersonal meaning relies heavily on verbal resources, and in particular on features of voice quality. The resource of voice quality will be explored in this paper, in terms of the contribution to the interpersonal meaning potential in a sample of problematic calls within the financial and insurance sectors.
Two of the major theoretical understandings informing this paper are those of the semantics of voice quality (van Leeuwen, 1999) and of Appraisal theory (Martin and White, 2005). Drawing on van Leeuwen’s categories, key voice quality features include those of tension, loudness, pitch register, vibrato and breathiness, etc. However, in analyses of the functioning of voice quality in the call centre discourse, we focus on the relative expression rather than categorical distinctions in these features. Voice quality features, as a kind of phonological feature, are inherently gradable, and we interpret the relative nature of the feature attended to. From the perspective of Appraisal theory we consider the attitudinal potential of the features. However, the attitude constructed by the CSR and/or customer is not simply viewed as individual expression, but rather as a resource for managing successful negotiation and forming alignment between the two parties. The findings from this study are twofold 1) they explore the boundaries of appraisal theory to include voice quality; and 2) at a more practical level, could be used to support the training and services offered by this rapid developing industry.
Martin, J. R., & White, P. R. R. (2005). The Language of Evaluation: appraisal in English. Great Britain:Antony Rowe Ltd, Chippenham and Eastbourne.
van Leeuwen, T. (1999). Speech, music, sound. London: Palgrave Macmillan.
Soundtrack clichés as semantic resource - film music “meanings” and their use in television commercials
Peter White
This paper will attempt to provide something of a stocktaking of the “meanings” which it is possible to convey musically as a result of the conventions and clichés of film music scoring. It will be proposed that these conventions make it possible for both experiential and attitudinal meanings to be conveyed musically and an attempt will be made to document some of the more obvious and widely used of these. Some discussion of the musical morphology/phonology of these clichés will be provided, although this will not be the primary focus of the paper. A major component of the paper will be an examination of the use made of these “meanings” in television commercials for both ironic and non-ironic purposes.
Interacting with hypertext: negotiational resources of hypertext that sustain inquiry
Katina Zammit, UWS
In a classroom environment, students often work in groups in order to obtain information from a hypertext program on a topic set by their teacher. Deciding where to go through a hypertext program is a complex task involving the intersection of user/s with the program and user/s with each other. What students construct through these negotiations is a text, or 'pathway', shaped through different meaning making resources, which they and the teacher hope will provide them with information. The text created, therefore, is a product of interaction/ negotiation. The resource of NEGOTIATION presented in this paper is used to account for and describe the interaction between a student user and a hypertext, the other student user/s and the student user in control of the mouse.
Using the analogy of casual conversation (Eggins & Slade, 1997), this paper will focus on describing the negotiational resources of hypertext. This description will be limited to sustaining moves. A description of sustaining moves reveal how it is the student user group moves onward, in an information-related way, through a hypertext program, building their own pathway. The description of these moves is good place to start examining how interaction/negotiation supports students' joint text building work. Within the sustaining moves, the paper will specifically focus on the sustaining: continue moves. Of the sustain:continue moves I will concentrate on two different types: the prolonging and monitor moves. Prolonging moves are an example of screen internal (S.I.) user actions and represent one way to develop information on a topic. monitor moves are an example of a continue move involving screen external (S.E) user actions.
Examples will be drawn from my doctoral research transcribed using a multimodal transcription method.
Eggins, S., & Slade, D. (1997). Analyzing Casual Conversation. London: Cassell.
Annotating and Visualising Co-instantiation
Michele Zappavigna, University of Sydney
Because of the complexity of texts, linguists require synoptic overviews that help them to perceive patterns such as couplings (Martin; Knight, 2007) and syndromes. A coupling is the co-instantiation of different discourse semantic or lexicogrammatical systems, while a syndrome is the tendency of a text to manifest a particular patterning of couplings. Systemic Functional Linguistics (SFL) theorises texts as multistratal and multidimensional, representing choices made from meaning potential as system networks. System networks are essentially tree diagrams turned on their side. One problem with this form of visualisation is that it is not very successful in representing complementarity. In other words, system networks do not support the representation of couplings and syndromes in individual texts or corpora. This paper introduces two software applications: Appraiser, a tool for annotating evaluative language in text, and, SysVis, a tool for exploring the patterning of texts in a 3D globe. The former aims to reduce the labour of annotation and provide a means of marking-up texts that may be visualised with the latter.
Restoring time factor in hypertext description: towards a dynamic model for Ideation Movement in children’s multimodal educational interactives
Sumin Zhao, Department of Linguistics, University of Sydney
The proposed paper raises the issue of ‘dynamism’ in language and multimodal description and modelling. It attempts to problematise the traditional model of text dynamism in which logogenetic process is decomposed into a series of a separate static elements. It argues that the first step in overcoming statics is to restore time order in text description. The restoration of time factor involves at least three fundamental theoretical re-conceptualisations: 1) the understanding of time and space as continuum in the observation of semiotic phenomenon, 2) the establishment of time-space metric for text description based on the existing theoretical parameters (logogenesis, instantiation and metafunction in particular) of Systemic Functional Linguistics theory, 3) the adaptation of linguistic description of logogenetic movements into 3-D computational model through visualisation technology. To illustrate the possibility of transforming these abstract theoretical concepts into analytical practices, the paper will exam in details the ideational relations (verbiage as well as image) and the construing of fields in a web-based children’s educational game—Medical emergency at Lonely Creek—from the National Museum of Australia. A preliminary manual 3-D visualisation of lexical movements through the logogenetic unfolding of the game will be presented. The paper will demonstrate the ways in which such visual presentation may assist in understanding the re-contextualisation of knowledge through field projection in pedagogy discourse as well as the concept of reading path in hypermedia environment. Ultimately, it is anticipated that the paper would address the dilemma the semioticians are facing while searching for new metalanguage to capture the complexity of human semiosis.
Reference
Baldry, A & Thibault, P. (2006) Multimodal transcription and text analysis. London: Equnox.
Bernstein, B (1990) The structuring of pedagogic discourse: Class, code and control (Vol.4). London and New York: Routledge.
Christie, F (2002) Classroom discourse analysis: A functional perspective. New York & London: Continuum
Halliday, M. A. K., and Matthiessen, C. M. I. M. (1999) Construing experience through meaning: A language-based approach to cognition. London: Cassell.
Halliday, M. A. K., and Matthiessen, C. M. I. M. (2004) An introduction to functional grammar 3rd ed. London: Arnold.
Hjelmslev, L. (1961) Prolegomena to a Theory of Language. Madison, Wisconsin: University of Wisconsin Press
Jakobson, R. (1990) On Language. Cambridge, Mass.: Harvard University Press.
Kress, G & van Leeuwen, T (2006) Reading images: the grammar of visual design. 2nd ed. London: Routledge.
Lemke, J (2002) ‘Travels in hypermodality.’ Visual Communication 1.3. 299-325.
Martin, J. R. (1985) ‘Process and text: two aspects of human semiosis.’ in J. D. Benson & W. S/ Greaves [Ed.] Systemic perspectives on discourse: selected theoretical papers from the 9th International Systemic Workshop. Norwood, N.J.: Ablex (Advances in Discourse processes 15)
Martin, J. R. (1991) ‘Intrinsic functionality: implications for contextual theory.’ Social Semiotics 1.1. 99-162.
Martin, J. R. (1992) English Text: system and structure. Amsterdam: Benjamins.
Martin, J. R . (1995) “Text and clause: fractal resonance”. Text 15.1: 5-42.
Martin, J. R. (1996) ‘Types of structure: deconstructing notions of constituency in clause and text’. E. H. Hovy & D. R. Scott [Ed.] Computational and Conversational Discourse: burning issues - an interdisciplinary account. Heidelberg: Springer
Martin, J. R. (In press) ‘Genre, ideology and intertextuality: a systemic functional perspective.’ Human Communication (Special Issue on Genre edited by J
Bateman)
Matthiessen, C.M.I.M (1988) ‘Representational issues in systemic functional grammar’ in J.D Benson & W. S. Greaves, Systemic functional perspectives on discourse. Norwood, NJ: Albex
O’ Donnell, M (1990) ‘A dynamic model of exchange.’ Word. 41(3), 293-327.
O’ Halloran, K (2005) Mathematical discourse: language, symbolism and visual images. London & New York: Continuum
Reichenbach, H (1957) The philosophy of space and time. New York: Dover Publications
Thibault, P (2000) ‘The multimodal transcription of television advertisement: theory and practice’ in A. Baldry [Ed.] (2000) Multimodality and multimediality in the distance learning age. Campo Basso: Lampo.
Ventola, E. (1987) The structure of social interaction: A systemic approach to the semiotics of service encounters. London: Frances Pinter.