PLM21015 Thematic session: Multimodal communication (meaning-making beyond language)

Conveners: Katarzyna Molek-Kozakowska (Opole University) and Małgorzata Haładewicz-Grzelak (Opole University of Technology)

Interrelations between various aspects of human activities within what Juri Lotman (1984[2005]) named ‘semiosphere’ have been explored by generations of scholars, from classical structuralist research such as Roman Jakobson’s (1985) and Boris Uspienski’s (1975) to contemporary works on multimodality by Gunther Kress and Theo van Leeuwen (1996), phonology as human behavior (Tobin, 1997), visual communication (Machin and Jaworski, 2006), or paralanguage and non-verbal communication (cf. Gibbon, 2011; Karpiński, 2012). Following this, our session aims to investigate communicative practices and products which link language with other modes of meaning-making.

This interrelation can be twofold. On the one hand, there is the realm of non-verbal communication, mostly within face-to-face interactions (gesture, eye contact, proxemics, etc.), where language seems to be the primary semiotic code (Halliday, 1978). However, in this session, multimodality can be also understood broader than that, closer to Barthes’ (1977) or Baudrillard’s (e.g. 2001) ideas of semiotic systems other than language itself possessing structural regularities able to carry communicative load. In other words, we aim at looking at meaningful communicative  relations outside the spoken/written language and the verbalized.

In accordance with the conference theme, we want to explore how linguistic concepts could be understood and applied in the study of multimodal communication. We also welcome new (multidisciplinary) research frameworks which could help to grasp multimodality analytically. A vast range of linguistic categories have already been productively used to explain the meaning-making mechanisms in multimodal communication, whereas concepts from outside linguistics have been incorporated into the theoretical framework of multimodal analysis to make such studies more in-depth and insightful (e.g. contributions in Jewitt, 2011). That is why, apart from strictly linguistic frameworks, we encourage discursive, semiotic and anthropological analyses of all sorts of multimodal data: verbal, visual, paralinguistic and audio (including artistic discourse).

The main objectives are to: 1) provide a forum for discussion on linguistic concepts in communication across modes, channels and media; 2) consider methodologies that allow studying language vis-à-vis other modes; 3) trace new qualities emergent through multimodal crisscrossing. 4) survey interrelations of various facets of multimodal cultural space; and 5) interrogate latest changes these might be undergoing. Possible foci include:

  • verbal and non-verbal communication (e.g., gestures, proxemics);
  • culture as highly structured  text (e.g., space, public display);
  • iconic and visual communication (e.g., imagery and symbols);
  • textuality  / orality in new media (e.g. multimodal ensembles);
  • multimodal corpora in research;
  • multimodal rhetoric(s);
  • media / visual literacy.

Submission deadline for this session: 22 March 2015.

Sample references:

Baudrillard,  Jean (2001). Selected writings. Stanford: Stanford University Press. 

Barthes, Ronald. (1977). Image-music-text. London: Fontana.

Gibbon, Dafydd. (2011). “Modelling gesture as speech: A linguistic approach”. In: Poznań Studies in Contemporary Linguistics. 470-508. 

Halliday, M. A. K. (1978). Language as a social semiotic. London: Edward Arnold.

Jakobson, Roman. (1985). Verbal art, verbal sign, verbal time. Minnesota: University of Minnesota Press. 

Jewitt, Carey (ed.). (2011). The Routledge handbook of multimodal analysis. London and New York: Routledge.

Karpiński, Maciej. (2012). “The boundaries of language: Dealing with paralinguistic features”. Lingua Posnaniensis, vol. LIV (2): 37-54. 

Kress, Gunther and Theo van Leeuwen. (1996). Reading images: The grammar of visual design. London: Rutledge

Lotman, Yuri. (1977[1973]). 1976. Semiotics of Cinema. [Transl. by Mark Suino.] Ann Arbor: University of Michigan Press.

Lotman, Yuri. 2005[1984]). “On the semiosphere”. In: Sign System Studies 33(1). [transl. Wilma Clark]. 205-229.

Machin, David and Adam Jaworski. (2006). “Archive video footage in news: creating a likeness and index of the phenomenal world”. Visual Communication 5(3): 345–366.

Norris, Sigrid. 2011. Identity in (Inter)action: Introducing Multimodal (Inter)action Analysis.  Berlin, Boston: deGruyter.

Norris, Sigrid and Maier, Carmen. D. (eds). 2014. Interactions, texts and images: A reader in multimodality. Berlin, Boston: de Gruyter Mouton.

Tobin, Yishai. (1997). Phonology as human behavior: Theoretical implications and clinical applications. Durham, NC/London: Duke University Press.

Unsworth, Len and A. Thomas (eds.). (2014). English teaching and New Literacies pedagogy: Interpreting and authoring digital multimedia narratives. New York: Peter Lang Publishing.

Uspienski, Boris. (1974[1973]). “Left and right in icon painting”. In: Semiotica. 13(1). 33-39. 

Uspienski, Boris= Успенский, Борис. (2007). Ego loquens. Язык и коммуникационное пространство. Moscow:  РГГУ. 

Yazdani, Masoud and Philip Barker (eds.). (2000). Iconic communication. Bristol: Intellect LTD.