---
output: rENA:::methods_report
---
`r ub = enaset[["_function.params"]]$units.by`
`r meta = enaset$meta.data[, ub, with = F]`
`r u = enaset$enadata$units`
`r cvb = enaset[["_function.params"]]$conversations.by`
`r cv = enaset[["_function.params"]]$conversation`
`r cvEx = as.vector(as.matrix((enaset$model$raw.input[1,cvb,with=F]))[1,])`
`r codes = enaset$rotation$codes`
`r wb = enaset[["_function.params"]]$window.size.back`
`r wgt = enaset[["_function.params"]]$weight.by`
`r normed = identical(enaset[["_function.params"]]$norm.by, rENA:::fun_sphere_norm)`
`r corr = ena.correlations(enaset)`
`r corrRnd = apply(apply(matrix(corr), 1, unlist), c(1,2), round, sig.dig)`
`r if(theory == T) paste0("ENA Theory")`

`r if(theory == T) paste0("Epistemic Network Analysis (ENA) is a quantitative ethnographic technique for modeling the structure of connections in data. ENA assumes: (1) that it is possible to systematically identify a set of meaningful features in the data (Codes); (2) that the data has local structure (conversations); and (3) that an important feature of the data is the way that Codes are connected to one another within conversations (Shaffer, 2017; Shaffer, Collier, & Ruis, 2016; Shaffer & Ruis, 2017). For example, if a team is working on a design project, they talk about important Codes such as production processes, design specifications, budget, and so on. They have a series of conversations at design meetings, and a key part of understanding their design process is modeling how they think about the relationships between production processes, specifications, budget, and other key parts of their design work (Arastoopour, Shaffer, Swiecki, Ruis, & Chesler, 2016). In an interview transcript, each answer to a question might be a unique conversation (Eagan & Hamilton, 2018), or in a collection of documents, each document or section of a document (Ruis, In press). ENA models the connections between Codes by quantifying the co-occurrence of Codes within conversations, producing a weighted network of co-occurrences, along with associated visualizations for each unit of analysis in the data. Critically, ENA analyzes all of the networks simultaneously, resulting in a set of networks that can be compared both visually and statistically.")`

`r if(theory == T) paste0("[IF A NEW FIELD FOR ENA MODELS:")`
`r if(theory == T) paste0("ENA was originally developed to model theories of cognition, discourse, and culture which argue that the connections people make in discourse are a critical level of analysis (Shaffer et al., 2009). DiSessa (1988), for example, characterized learning as a process by which isolated elements of experiential knowledge are connected through theoretical frameworks to develop both new knowledge and deep, systematic understanding. Similarly, Linn and colleagues (2004) argue that learners develop STEM expertise by constructing a knowledge web: a repertoire of ideas and the connections among them. Shaffer (2006, 2007, 2012) characterized learning as the development of an epistemic frame: a pattern of connections among knowledge, skills, habits of mind, and other cognitive elements that characterize communities of practice (Hutchins, 1995; Shaffer, 2004; Wenger, 1999), or groups of people who share similar ways of framing, investigating, and solving complex problems.")`

`r if(theory == T) paste0("While ENA was originally designed to address challenges in learning analytics, the method is not limited to analyses of learning data. For example, ENA has been used to analyze (a) surgery trainees’ operative performance during a simulated procedure (Ruis et al., in press); (b) gaze coordination during collaborative work (Andrist, Collier, Gleicher, Mutlu, & Shaffer, 2015); and (c) communication among health care teams (Sullivan et al., 2018; Wooldridge, Carayon, Eagan, & Shaffer, 2018). The key assumption of the method is that the structure of connections in the data is the most important in the analysis. In other words, ENA is an appropriate technique for any context in which the structure of connections is meaningful.","\n","ENA is thus a useful technique for modeling [your topic] because it can model the relationships among [features of interest in your data] as they occur within [your conversations].]")`

`r if(methods == T) paste0("ENA Methods")`

`r if(methods == T) paste0("In this study, we applied Epistemic Network Analysis (Shaffer, 2017; Shaffer, Collier, & Ruis, 2016; Shaffer & Ruis, 2017) to our data using the ENA", if(tool=="rENA") "package for the statistical programming language R (version ", tool.version, ") (Marquart, Swiecki, et al., 2018)", if(tool=="webENA") " Web Tool (version ", tool.version, ") (Marquart, Hinojosa, Swiecki, Eagan, & Shaffer, 2018)")`

`r if(methods == T) paste0("We defined the units of analysis as all lines of data associated with a single value of ", ub[1], if(length(ub) > 1)" subsetted by ", concatenate::cc_and(ub[2:length(ub)], oxford=T), ". For example, one unit consisted of all the lines associated with ", concatenate::cc_and(apply(matrix(c(colnames(meta)[-1], t(meta[1,-c(1)])), ncol = 2, nrow = ncol(meta)-1), 1, paste, collapse = " "), oxford = T),".")`

`r if(methods == T) paste0( if(identical(wb, "Conversation")) "The ENA algorithm constructs a network model for each unit of analysis, showing how the codes in all lines associated with that unit—within a given set of lines called a conversation—are connected to one another. The resulting networks are aggregated for all conversations for each unit of analysis in the model." else if(is.numeric(wb)) paste0("The ENA algorithm uses a moving window to construct a network model for each line in the data, showing how codes in the current line are connected to codes that occur within the recent temporal context (Siebert-Evenstone et al., 2017), defined as ", (if(!is.infinite(wb)){ paste0(wb," lines (each line plus the ",(wb-1)," previous lines)")} else {("all lines preceding the current line")}))," within a given conversation. The resulting networks are aggregated for all lines for each unit of analysis in the model. In this model, we aggregated networks using a ",if(identical('binary',wgt)){("binary")}else{("weighted")},(" summation in which the networks for a given"),(if(identical(wb, "Conversation")){(" conversation")}else{(" line")}),(" reflect"),(if(identical('binary',wgt)){(" the presence or absence of the co-occurrence")} else if(identical(wgt, 'product')){(" the product of the value")} else if(identical(wgt,'sqrt')){(" square root of the product")} else{ (" log of the product") }),(" of each pair of codes."))`

`r if(methods == T) paste0("Our ENA model included the following codes: ", concatenate::cc_and(codes), ". We defined conversations as all lines of data associated with a single value of ", cvb[1], if(length(cvb) > 1) paste0(" subsetted by ", concatenate::cc_and(cvb[2:length(cvb)], oxford=T)), ". For example, one conversation consisted of all the lines associated with ", concatenate::cc_and(apply(matrix(c(cvb, cvEx), nrow=2), 1, paste, collapse=" "), oxford=T),".")`

`r if(methods ==T) paste0(if(normed) paste0("The ENA model normalized the networks for all units of analysis before they were subjected to a dimensional reduction, which accounts for the fact that different units of analysis may have different amounts of coded lines in the data. "),"For the dimensional reduction, we used a ", if(identical(enaset$function.params$rotation.by, rENA::ena.rotate.by.mean)) paste0("means rotation based on two groups in our data: ", concatenate::cc_and(names(enaset$function.params$rotation.params)),". This projection highlights the differences between these groups (if any) by constructing a dimensional reduction that places the means of the groups as close as possible to the x-axis of the projected space. Subsequent dimensions were projected using")," singular value decomposition, which produces orthogonal dimensions that maximize the variance explained by each dimension. (See Shaffer et al., 2016 for a more detailed explanation of the mathematics; see Arastoopour, Swiecki, Chesler, & Shaffer, 2015 and Sullivan et al., 2017 for examples of this kind of analysis.)")`

`r if(methods ==T) paste0("Networks were visualized using network graphs where nodes correspond to the codes, and edges reflect the", if(normed) " relative"," frequency of co-occurrence, or connection, between two codes. The result is two coordinated representations for each unit of analysis: (1) a plotted point, which represents the location of that unit’s network in the low-dimensional projected space, and (2) a weighted network graph. The positions of the network graph nodes are fixed, and those positions are determined by an optimization routine that minimizes the difference between the plotted points and their corresponding network centroids. Because of this co-registration of network graphs and projected space, the positions of the network graph nodes—and the connections they define—can be used to interpret the dimensions of the projected space and explain the positions of plotted points in the space. Our model had co-registration correlations of ", corrRnd[1,1]," (Pearson) and ",corrRnd[1,2]," (Spearman) for the first dimension and co-registration correlations of ", corrRnd[2,1]," (Pearson) and ",corrRnd[2,2]," (Spearman) for the second.")``r if(methods == T && all(corrRnd > 0.9)) paste0(" These measures indicate that there is a strong goodness of fit between the visualization and the original model.")`

`r if(methods == T && !is.null(comparison)) paste0("ENA can be used to compare units of analysis in terms of their plotted point positions, individual networks, mean plotted point positions, and mean networks, which average the connection weights across individual networks. Networks may also be compared using network difference graphs. These graphs are calculated by subtracting the weight of each connection in one network from the corresponding connections in another.")`

`r if(methods == T && !is.null(comparison) && !is.null(comparison.groups)) paste0("To test for differences we applied a ", if(comparison == "parametric") "two-sample t-test assuming unequal variance" else "Mann-Whitney test", " to the location of points in the projected ENA space for units in ", comparison.groups[1], " and ", comparison.groups[2], ".")`

References

`r if(methods == T) paste0("Arastoopour, G., Swiecki, Z., Chesler, N. C., & Shaffer, D. W. (2015). Epistemic Network Analysis as a tool for engineering design assessment. Presented at the American Society for Engineering Education, Seattle, WA.")`

`r if(theory == T) paste0("Andrist, S., Collier, W., Gleicher, M., Mutlu, B., & Shaffer, D. (2015). Look together: Analyzing gaze coordination with epistemic network analysis. Frontiers in Psychology, 6(1016).")`

`r if(theory == T) paste0("Arastoopour, G., Shaffer, D. W., Swiecki, Z., Ruis, A. R., & Chesler, N. C. (2016). Teaching and assessing engineering design thinking with virtual internships and epistemic network analysis. International Journal of Engineering Education, 32(3B), 1492–1501.")`

`r if(theory == T) paste0("DiSessa, A. A. (1988). Knowledge in pieces. In G. Forman & P. Pufall (Eds.), Constructivism in the computer age (pp. 47–70). Hillsdale, NJ: Erlbaum.")`

`r if(theory == T) paste0("Eagan, B., & Hamilton, E. (2018). Epistemic Network Analysis fo an International Digital Makerspace in Africa, Europe, and the US. Presented at the Annual Meeting of the American Education Research Association, New York, NY.")`

`r if(theory == T) paste0("Hutchins, E. (1995). Cognition in the wild. Cambridge, MA: MIT Press.")`

`r if(theory == T) paste0("Linn, M. C., Eylon, B.-S., & Davis, E. A. (2004). The knowledge integration perspective on learning. In M. C. Linn, E. A. Davis, & P. Bell (Eds.), Internet environments for science education (pp. 29–46). Mahwah, NJ: Lawrence Erlbaum Associates.")`

`r if(methods == T && tool=="webENA") paste0("Marquart, C. L., Hinojosa, C., Swiecki, Z., Eagan, B., & Shaffer, D. W. (2018). Epistemic Network Analysis (Version ", tool.version, ") [Software]. Available from http://app.epistemicnetwork.org")``r if(methods == T && tool=="rENA") paste0("Marquart, C. L., Swiecki, Z., Collier, W., Eagan, B., Woodward, R., & Shaffer, D. W. (2018). rENA: Epistemic Network Analysis (Version ", tool.version, "). Retrieved from https://cran.r-project.org/web/packages/rENA/index.html")`

`r if(theory == T) paste0("Ruis, A. R. (2018). \"Trois Empreintes d’un Même Cachet\": Toward a Historical Definition of Nutrition. In E. T. Ewing & K. Randall (eds.), Viral networks: Connecting digital humanities and medical history (pp. 179–212). Blacksburg: VT Publishing.")`

`r if(theory == T) paste0("Ruis, A. R., Rosser, A. A., Quandt-Walle, C., Nathwani, J. N., Shaffer, D. W., & Pugh, C. M. (2018). The hands and head of a surgeon: Modeling operative competency with multimodal epistemic network analysis. American Journal of Surgery, 216(5), 835-840.")`

`r if(theory == T) paste0("Shaffer, D. W. (2004). Pedagogical praxis: Using technology to build professional communities of practice. Association for Computing Machinery (ACM) SigGROUP Bulletin, 24(3), 39–43.")`

`r if(theory == T) paste0("Shaffer, D. W. (2006). Epistemic frames for epistemic games. Computers and Education, 46(3), 223–234.")`

`r if(theory == T) paste0("Shaffer, D. W. (2007). How computer games help children learn. New York, NY: Palgrave Macmillan.")`

`r if(theory == T) paste0("Shaffer, D. W. (2012). Models of situated action: Computer games and the problem of transfer. In C. Steinkuehler, K. D. Squire, & S. A. Barab (Eds.), Games, learning, and society: Learning and meaning in the digital age (pp. 403–431). Cambridge, UK: Cambridge University Press.")`

Shaffer, D. W. (2017). Quantitative ethnography. Madison, WI: Cathcart Press.

Shaffer, D. W., Collier, W., & Ruis, A. R. (2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. Journal of Learning Analytics, 3(3), 9–45.

`r if(theory == T) paste0("Shaffer, D. W., Hatfield, D. L., Svarovsky, G. N., Nash, P., Nulty, A., Bagley, E. A., … Frank, K. (2009). Epistemic network analysis: A prototype for 21st century assessment of learning. International Journal of Learning and Media, 1(1), 1–21.")`

Shaffer, D. W., & Ruis, A. R. (2017). Epistemic network analysis: A worked example of theory-based learning analytics. In C. Lang, G. Siemens, A. F. Wise, & D. Gasevic (Eds.), Handbook of learning analytics (pp. 175–187). Society for Learning Analytics Research.

`r if(methods == T) paste0("Siebert-Evenstone, A., Arastoopour Irgens, G., Collier, W., Swiecki, Z., Ruis, A. R., & Williamson Shaffer, D. (2017). In Search of Conversational Grain Size: Modelling Semantic Structure Using Moving Stanza Windows. Journal of Learning Analytics, 4(3), 123–139. https://doi.org/10.18608/jla.2017.43.7")`

Sullivan, S. A., Warner-Hillard, C., Eagan, B. R., Thompson, R., Ruis, A. R., Haines, K., … Jung, H. S. (2017). Using epistemic network analysis to identify targets for educational interventions in trauma team communication. Surgery, in press.

`r if(theory == T) paste0("Wenger, E. (1999). Communities of practice: Learning, meaning, and identity. Cambridge, UK: Cambridge University Press. Retrieved from http://books.google.com/books?id=heBZpgYUKdAC&pgis=1")`

`r if(theory == T) paste0("Wooldridge, A. R., Carayon, P., Eagan, B. R., & Shaffer, D. W. (2018). Quantifying the qualitative with epistemic network analysis: A human factors case study of task-allocation communication in a primary care team. IIE Transactions on Healthcare Systems Engineering, in press.")`
