Before You Tie the Knot:
Mapping Pedagogy, Learning Outcomes, and Effect Size
In Premarital Education
Human services educators are continually seeking ways to make instruction more effective and engaging. This study evaluated the AIAI-FTFD start-to-finish teaching model for educators in an ongoing premarital educational program to determine the effectiveness of this model in implementing the concept of “teaching as an intervention” in human services educational programming. Specifically, this implementation science study assessed the targeted cognitive, emotional, and behavioral learning outcomes generated by using the AIAI-FTFD teaching model while completing the Before You Tie the Knot (BYTK) premarital education program online. A self-reported quantitative evaluation design was utilized to assess key objectives in the sample (n=93). Clearly evident effect sizes were found in perceived knowledge gain and perceived confidence gain in the ability to implement the skills covered in the training. Subsequent discussion focuses on how the AIAI-FTFD start-to-finish teaching model can facilitate change and learning in educational settings.
Keywords: teaching, effective teaching, premarital education, communication, human services
Before You Tie the Knot:
Mapping Pedagogy, Learning Outcomes, and Effect Size in Continuing Education
Using intentional and sound pedagogical practices is critical to maximizing the change process in an educational learning environment (Cole, 1981; Mace, 1981; Powell & Cassidy, 2007; Stevenson & Harris, 2014). Catching the learners’ attention, introducing new information, facilitating interaction between the teacher and the learners, and providing experimental methods for the learners to apply and practice targeted cognitive, emotional, and behavioral learning skills, both during and after educational programming, has been shown to maximize learning outcomes (Edgar, 1969; XXXX et al., 2014; Merrill, 1997). Often, in the absence of intentional teaching practice and sound pedagogy, too much information is presented with too little time spent on applying and practicing target skills. Cognitive overload, a situation in which a learner is presented with too much information at once, may inhibit an individual from successfully learning the core concepts being taught (Paas, Renkl, & Sweller, 2004). Conversely, active learning (in-class activity, application, and participation), as opposed to more-straightforward lecture techniques, has been shown to result in higher student gains on class-specific outcomes (Hackathorn, Solomon, Blankmeyer, Tennial, & Garczynski, 2011; Michel, Cater, & Varela, 2009).
One antidote to lecturing is facilitating the process of teaching less better by focusing on a few target skills and carefully evaluating resultant learning outcomes (XXXX et al., 2014). At the individual level, teaching approaches exist to implement this strategy in a variety of learning environments. The AIAI-FTFD (Attention, Interact, Apply, Invite – Fact, Think, Feel, Do) Start-to-Finish Teaching Model is a teaching tool that can be used in many subject areas to practice the idea of teaching less better. This teaching model was specifically applied to a premarital preparation training in this study and then evaluated based on the participants’ reported training outcomes. Through examining pertinent background information and application of this model, further insight into implementation science can be gained to potentially inform future approaches to premarital education teaching and programming.
A review of the AIAI-FTFD model, its theoretical foundation, and the curriculum in which it is applied, Before You Tie the Knot, are provided here; general summaries of the model and its theoretical underpinnings are also summarized in previous AIAI-FTFD related research (XXXX, Chartier, & Davis, 2010; XXXX et al., 2014; XXXX, XXXX, & Schmeer, 2016).
Teaching as an Intervention- Before You Tie the Knot: Mapping Pedagogy, Learning Outcomes, and Effect Size
Previous research has demonstrated that effective teaching methods must include at least the following: assessing learner needs and addressing these specific needs in the teaching environment; founding teaching practices on theory-based and empirically-informed methodologies; understanding, negotiating, and managing learners and group processes successfully; and, realistically evaluating the teaching experience (Gagné, Briggs, & Wager, 1992; Latham, 2002; Powell & Cassidy, 2007). Before any lesson begins, instructors should consider what the learner will need in order to accomplish these goals and identify specific cognitive, emotional, and behavioral target skills (i.e., learning outcomes). Related approaches may also include identifying which higher-order thinking (cognitive) skills will be covered in the program or lesson (Anderson & Krathwohl, 2001; Bloom, Engelhart, Furst, Hill, & Krathwohl, 1956). It is virtually impossible to evaluate and measure these learning outcomes if they are not identified prior to teaching. Knowing the learners and their felt, ascribed, and future needs at the outset (Powell & Cassidy, 2007) allows the instructional outline to be specifically tailored to the learners, thus maximizing the potential for positive learning outcomes.
Once learners’ needs have been assessed and specific learning outcomes have been identified, establishing clear learner-centered objectives and goals are essential to guiding the teaching preparation and delivery process (Gagné, Briggs, & Wager, 1992; Bennett & Rockwell, 1995). Clarifying and determining the instructor and learner objectives and goals informs best practice instructional designs for content mastery, understanding, and application (XXXX et al., 2014; Merrill, 1991; Merrill, 1997). Objectives can also help instructors focus the lesson, assess the effectiveness of instruction, and point toward opportunities for improvement in future training (Tyler, 1949). Additional best practices, such as those compiled by Rosenshine (1983), emphasize the importance of structure, examples, feedback, and, ultimately, opportunities for continued practicing of the identified knowledge and skill learning outcomes. The overall aim of a teaching outline should be to shape the content and instructional techniques into an intentional lesson plan for how to engage the learner and to maximize learning outcomes (Gagné, Briggs, & Wager, 1992; XXXX et al., 2014; Wiggins & McTighe, 2005).
The AIAI-FTFD Teaching Model Before You Tie the Knot: Mapping Pedagogy, Learning Outcomes, and Effect Size
The AIAI-FTFD Start-to-Finish Teaching Model (Figure 1) is designed as an instructional tool that can be used across a diverse set of topics and contexts to improve instruction and learning outcomes (XXXX et al., 2014). The model conceptualizes principles of effective teaching in a systematic, step-by-step, start-to-finish format outlining specific preparation and delivery procedures (Gagné, Briggs, & Wager, 1992; XXXX Chartier, & Davis, 2010; XXXX & Lee, 2006). The primary foci of the AIAI-FTFD teaching model include initially assessing the learner’s (audience’s) needs and then targeting learning outcomes measured by cognitive, emotional (e.g., confidence, attitudes), and/or behavioral skills that the instructor identifies as important to the learning process. The first step in the AIAI-FTFD model, Attention, is designed specifically to catch the learners’ attention. Then, the model quickly moves to Interaction, the second step, in which the instructor engages learners with pertinent information and concepts. Information is communicated via different sensory modalities (i.e., visual, auditory, kinesthetic), primarily through facilitating discussion, except in certain contexts when lecture (or playing the expert role) is required. The instructor may also use the consultation role when learners are engaged individually or in groups with tasks (e.g., problem-based learning) that require the instructor to provide input when asked (Teemant, Moen, & XXXX, 2013; Powell & Cassidy, 2007).
The instructor then asks four kinds of specific, goal-directed questions (i.e., Fact, Think, Feel, Do) about the given topic to facilitate discussion and guides the learners to interact with the information, the instructor, and each other. Specifically, the Fact, Think, Feel, Do (FTFD) component of the teaching model includes a systematic series of questions instructors may pose to the learners to engage in higher level critical thinking and meaningful discussion. Research indicates that effective questioning promotes higher levels of thinking and improves overall retention of information learned (Edgar, 1969; Gagné, Briggs, & Wager, 1992).
Application, or applying the information learned, is the third step in the AIAI-FTFD teaching process. There is direct positive association between the amount of time spent on this step and positive learning outcomes (XXXX & Lee, 2006; XXXX et al., 2010). Application consists of encouraging learners to make practical applications of the principles and materials the presentation covers. Application also allows for learners to achieve new cognitive, emotional, and behavioral learning outcomes pertaining to the material taught. The AIAI-FTFD teaching model emphasizes the importance of taking intentional time to allow learners to practice these target skills. During the Invitation step, learners are able to continue to practice and track these skills at home. The invitation is often introduced in the form of homework and/or through the use of a tracking chart to evaluate ongoing progress for achieving the identified target skills (Badger, 2008; XXXX et al., 2014).Before You Tie the Knot: Mapping Pedagogy, Learning Outcomes, and Effect Size
The Preparation section in the top half of the AIAI-FTFD teaching model (Figure 1) requires instructors to create lesson plans by (a) assessing learners’ needs; (b) deciding on associated content; (c) determining cognitive, emotional, and behavioral target skills; (d) listing instructional objectives and overall learning goals; (e) identifying what the instructor and the learner will do to accomplish identified learning outcomes (i.e., target skills); and (f) determining the type of content, the mental processes that will be engaged, the method of delivery, and the general teaching roles instructors will play in executing this plan (e.g., expert, facilitator, or consultant) (XXXX et al., 2010; XXXX et al., 2014). The model also provides a specific method of instructional Delivery to implement this plan. Many methods of instruction are available, but few are organized into a start-to-finish, step-by-step model for preparing educators across disciplines to teach effectively (XXXX, 2009; XXXX, XXXX, & Schmeer, 2016).
THE AIAI-FTFD START-TO-FINISH INSTRUCTIONAL Model-before You Tie the Knot: Mapping Pedagogy, Learning Outcomes, and Effect Size
© XXXX, Ph.D.
Content 2-3 Concepts/Principles I will teach:
|Target Skills-Cognitive (knowledge), Emotional (confidence – attitude change), and Behavioral (skills) Processes:
1. Cognitive/Know (C) –
2. Emotional/Apply (E) –
3. Behavioral/Practice (B) –
|Objectives (mapped to target skills):
1. (C) – Participants will identify (know) . . .
2. (E) – Participants will apply . . .
3. (B) – Participants will practice . . .
Role: Expert, Facilitator, or Consultant (Circle One)
|Instructor Will Do
1. (C) Know
2. (E) Apply
3. (B) Practice
|Learner Will Do
1. (C) Know
2. (E) Apply
3. (B) Practice
This lesson will use:
This lesson will engage:
This lesson will use:
2. Visual 3. Praxis
|Delivery: Lesson Outline Role: Expert, Facilitator, Consultant|
|Attention: Question Types:
Practice Target Skills: Cognitive, Emotional, Behavioral (5-10 minutes)
Figure 1. AIAI-FTFD Start-to-Finish Conceptual Instructional Model-before You Tie the Knot: Mapping Pedagogy, Learning Outcomes, and Effect Size
Effectiveness of Premarital Preparation Programming-before You Tie the Knot: Mapping Pedagogy, Learning Outcomes, and Effect Size
In this study, the AIAI-FTFD model was applied in the context of an online premarital preparation educational program. Premarital preparation programs have generally shown mixed results in effectiveness. There is an abundance of evidence that premarital education and skills training results improved relationship quality and satisfaction; premarital education programs have been shown to be positively related to higher levels of dyadic couple relationship quality and satisfaction, lower levels of between-partner conflict, and lower rates of relationship dissolution (Fawcett, Hawkins, Blanchard, & Carroll, 2010; Rogge, Cobb, Lawrence, Johnson, & Bradbury, 2013; Stanley, Amato, Johnson, & Markman, 2006). It is not surprising, then, that domains frequently addressed by these programs — such as communication, conflict resolution, and type of inter-partner interaction — are also correlates of marital quality and satisfaction (Rogge et al., 2013; XXXX, Schramm, Marshall, & Lee, 2012). Consequently, premarital training has become more widely accessible over the past decade (Stanley et al., 2006). However, Doss, Rhoades, Stanley, Markman, and Johnson (2009) pointed out that couples at risk for relationship dissolution — those most in need — were less likely to access premarital education programs. Furthermore, Fawcett et al. (2010) questioned the results of much of this body of research. In a meta-analysis of forty-seven studies, these authors found that premarital training is not significantly related to dyadic couple relationship quality and satisfaction when both the published and unpublished research reports are included (such as doctoral dissertations), and found no evidence supporting a significant positive relationship between premarital education and relationship quality over the period of time typically addressed by these studies.
These modest effects of educational programming on subsequent dyadic couple relationship quality and satisfaction illustrate a real need for improvement in this area, particularly through the development and inclusion of pedagogical, programmatic, and evaluation methodologies whereby couples incorporate communication and conflict resolution skills into the challenges of day-to-day living (Fawcett et al., 2010). These authors suggest that social scientists must “critically examine and reconsider the content, intensity, methods, settings, delivery mechanisms, and target populations of premarital education” (p. 236).
To this end, the AIAI-FTFD start-to-finish programmatic and instructional model is proposed as an educational tool critically designed to assist educators with content, intensity, methods, settings, delivery mechanisms, and target populations as they disseminate information and skills training premarital couples can use to be successful in their day-to-day relationships. In the current study, the model’s effectiveness as an instructional tool has been evaluated through its use in teaching the Before You Tie the Knot premarital preparation program.
Premarital preparation requirements in state laws. In many states where premarital programming occurs, explicit statutes require premarital programs to meet specific content requirements and/or to demonstrate program effectiveness, such as in Georgia, Maryland, Minnesota, Oklahoma, Tennessee, Texas, and Florida (Harrison, 2011). For example, one part of Florida Statute 741.0305 (Online Sunshine: The Official Internet Site for the Florida Legislature, 2016) requires the following in order for premarital couples to receive a fee reduction on their marriage licenses: “…(2) The premarital preparation course may include instruction regarding: (a) Conflict management. (b) Communication skills. (c) Financial responsibilities (d) Children and parenting responsibilities. (e) Data compiled from available information relating to problems reported by married couples who seek marital or individual counseling.” These topics represent key components of successful marital relationships.
Before You Tie the Knot Premarital Preparation Program-before You Tie the Knot: Mapping Pedagogy, Learning Outcomes, and Effect Size
Before You Tie the Knot is a research-based program that meets the requirements of Florida Statute 741.0305 and qualifies couples who complete the program for a reduction of the marriage license fee in designated counties. It is designed using the AIAI-FTFD instructional model to assist premarital couples to achieve relationship satisfaction and quality in their relationships by helping them to recognize their own and their partner’s needs, parent positively, negotiate conflict successfully, communicate effectively, manage money skillfully, and develop and maintain healthy lifestyles (SMARTcouples.org, 2016). It is one of four relationship education programs being delivered in Florida funded by the United States Department of Health and Human Services, Administration for Children and Families, Grant:#90FM0079-01-00.
Meeting personal and partner needs. Wallerstein (1996) asserts that marital happiness can be achieved through the perceived goodness of fit between individual and couple needs, wishes, and expectations. “Needs” are the requirements both individuals, families, and intimate partners have “that must be met at some level if they are to survive and engage in adaptive behavior” (Bubolz & Sontag, 2013, p. 435). These include physiological, social, emotional, and behavioral needs, all of which may be influenced by the human built, the social-cultural, and the natural physical-biological environmental ecosystems. Coplen and MacArthur (1982) identified at least eight categories of these needs that shape individuals, intimate partners, and their environments: to feel safe, to feel as though we belong, to develop a positive sense of personal identity, to have close real love relationships, to receive respect, to feel worthwhile, to feel capable (competent), and to experience growth.
The first module of the Before You Tie the Knot premarital preparation program assists couples in identifying these needs and helps them address their own and their partner’s needs in each of these eight categories. The AIAI-FTFD instructional model provides the methodology to help individuals and couples address meeting these needs through practice activities during this module and provides a tracking chart for them to continue to practice meeting their own and their partner’s needs outside of the classroom setting – a best practice in maximizing knowledge and skill development.
Children and parenting responsibilities. Responsible and positive parenting are associated with couple relationship satisfaction and stability (Gottman & Notarius, 2000; XXXX, 2010; XXXX, Johnson, & Olsen, 2013). Parental warmth, connectedness, and monitoring skills have been found to be effective in influencing short-term child outcomes of secure attachment, playful exploration and motivation, and effective communication and long-term child outcomes of healthy social-emotional, cognitive, and language ability development (Roggman, Boyce, & Innocenti, 2008). The second module of the Before You Tie the Knot premarital preparation program helps couples to identify and practice parental warmth, connectedness, and monitoring skills through the use of Latham’s Positive Parenting (1994) principles and the Love and Logic principles authored by Cline and Fay (2006). Tips for stepparents and co-parents are also included in this module (Allgood, Higgenbotham, & Skogrand, 2007a; 2007b). An invitation and a tracking chart to continue to practice the parenting skills learned with the children in the sphere of premarital couples’ lives is also provided in accordance with the AIAI-FTFD model.Before You Tie the Knot: Mapping Pedagogy, Learning Outcomes, and Effect Size
Conflict management and communication. Larson and Holman (1994) identified interactional processes (i.e., conflict management and communication) as the most predictive factors that influence relationship satisfaction and quality when compared with individual traits and contexts (Larson, 2003). Gottman, Coan, Carrere, and Swanson (1998) identified gentleness, soothing behaviors, and de-escalation of negativity as the key factors in positive interaction. Balance theory was cited as an explanation for the need to balance negative and positive interactions. According to Gottman (1994a), the optimal ratio of positive to negative interactions is 5:1. Gottman (1994a) has also specifically identified four negative behaviors that act as a deterrent to positive communication: criticism, defensiveness, contempt, and stonewalling. Five healthy communication and conflict resolution behaviors that promote positive interaction have also been identified (Gottman, 1994b): calm down, I-messages, speak non-defensively, validate, and overlearn the other eight skills. The Before You Tie the Knot premarital preparation program introduces the 9 Skills of Communication (9 Skills) (Gottman (1994a; 1994b) to couples along with 10 Rules for Conflict Constructive Conflict (10 Rules) (XXXX, 2012) in the third and fourth modules. Couples practice the 9 Skills and 10 Rules during the sessions and are provided with tracking charts so they can continue to practice these skills at home.
Financial management responsibilities. Levels of debt are highly associated with marital satisfaction and stability (Amato, Johnson, Booth, & Rogers, 2003; Dew, 2008). In fact, debt and financial issues constitute some of the biggest trouble spots identified by newlyweds (Schramm, Marshall, XXXX, & Lee, 2005) and by couples in general (XXXX, Schramm, Marshall, & Lee, 2012). Therefore, learning to manage finances in responsible ways is critical to relationship success. In the fifth module of the Before You Tie the Knot, understanding money habitudes, setting SMART financial goals, defining roles and responsibilities, putting together a plan for managing finances, and learning about the legalities of marriage in the financial world are addressed. The AIAI-FTFD teaching methodology is used to facilitate the capstone activity of this module, which includes assisting couples to develop a viable budget.
Healthy lifestyles. Getting married should not be a recipe for weight gain, but this is a reality for many couples in America (Hitti, 2007; The & Gordon-Larsen, 2009). Generally, however, marriage tends to influence men’s health in positive ways biologically, behaviorally, and psychologically (Harvard Medical School, 2010; Markey, Markey, & Gray, 2007). Men are not the only beneficiaries of health due to marriage; women’s health is also benefitted from marriage, but interestingly according to one study, only when they are in a satisfying marriage (DeNoon, 2003). Healthy lifestyle is the topic of Before You Tie the Knot’s sixth module. It includes understanding couples’ emotional, psychological, and physical health, the positive and negative impacts of relationships on health and vice versa, and learning ways to practice healthy lifestyles individually and as a couple. The AIAI-FTFD teaching model is used to promote wellness through managing stress, exercise, and nutrition in this module.
Objectives-before You Tie the Knot: Mapping Pedagogy, Learning Outcomes, and Effect Size
Transforming target skills into learning objectives is an important key to employing best practices in teaching (Gagné, Briggs, & Wager, 1992; XXXX et al., 2014). The objectives identified for this study using the AIAI-FTFD method correspond to core goals of the Before You Tie the Knot (BYTK) training. The objectives of the BYTK program include:
Objective 1. Participants will increase their levels of understanding (knowledge) about the factors associated with meeting their own and their partner’s needs, parenting effectively, healthy communication and conflict resolution patterns, managing money well, and practicing healthy lifestyles.
Objective 2. Participants will demonstrate increased changes in levels of confidence (attitudes) about their abilities to meet their own and their partner’s needs, parent effectively, communicate and resolve conflict in healthy ways, managing money well, and practice healthy lifestyles.
Objective 3. Participants will demonstrate positive skills (behaviors) to increase positive interaction, decrease negative interaction, increase positive bonds, and increase satisfaction and well-being, four primary indicators of healthy relationship stability and success, in each of the areas identified in Objectives 1 and 2 (XXXX, 2014; XXXX, Schramm, Marshall, & Lee, 2012).
Purpose-before You Tie the Knot: Mapping Pedagogy, Learning Outcomes, and Effect Size
The purpose of this study was to evaluate an ongoing premarital education program, designed using the AIAI-FTFD start-to-finish teaching model for human services educators (XXXX, et al., 2014), as a potential model for employing effective teaching as an intervention in human services educational programming. The research question that drove this exploratory study was, “What are the cognitive, emotional, and behavioral change learning outcomes generated by employing the AIAI-FTFD teaching model as an intervention in designing, delivering, and evaluating the Before You Tie the Knot (BYTK) program in an online learning environment?”
This study represents an expansion of previous studies of the AIAI-FTFD teaching model (XXXX, et al., 2010; XXXX, et al., 2014). The authors used a self-report quantitative evaluation method across an array of program contexts to study the effectiveness of the AIAI-FTFD teaching model in an educational learning environment among participants who completed the Before You Tie the Knot training.
The sample in this study was drawn from participants (n=93) in a Southeastern state who voluntarily completed a 6-hour online premarital education program titled, Before You Tie the Knot. A majority of subjects who participated in this study were White, female, below age 29, and single. Most participants made less than $40,000 a year or more than $80,000 per year and had a high school degree or higher (Table 1).
Missing demographic information is also reported in Table 1. Because this was an IRB-approved study, participants received a letter of information clearly informing them that participation in the program was strictly voluntary and that any survey item they did not want to complete was strictly at their discretion. Data used for from the BYTK variables was generally not included unless it was complete. In several cases, where only one or two data points were missing, the overall mean for the variable was calculated by reducing the n to the participants who had completed the questionnaire item and then averaging the overall scores to determine the overall mean.
Demographic Description of BYTK Participants (N = 93)
|70 and above||0||0|
|$80,000 or more||21||22|
|Less than high school||0||0|
|High school graduate/ GED||27||28|
Research and Curriculum Design and Delivery
The research design used for this IRB-approved study was a self-report quantitative exploratory cross-sectional design using a purposive sampling method. The BYTK curriculum used in this study was designed for human services environments in order to employ best practices in program design, implementation, and evaluation (Powell & Cassidy, 2007) using the AIAI-FTFD teaching model (XXXX et al., 2014). The AIAI-FTFD teaching model was included in the notes section of the BYTK PowerPoint used to deliver the curriculum with embedded accompanying teaching strategies, instructional methods, and questioning techniques. The curriculum was administered online in six one-hour sessions. Participants could stop and start the sessions, come back to them at any time, and continue their participation. Videos, lectures, activities, and quizzes were required to complete each session including a required 100% correct score on all quizzes, which could each be taken multiple times. A certificate of completion was awarded and could be downloaded after all requirements were successfully completed. No compensation was granted for participation in the study.
Data Collection and Analysis
A one-time retrospective pre-then-post online survey instrument (i.e., Qualtrics) was administered to assess participants’ knowledge, confidence, and behavior change at the end of the BYTK program. A five level Likert scale providing a range of responses (strongly agree, disagree, neither agree nor disagree, agree, and strongly disagree) was used to assess knowledge, confidence, and behavior change for the program variables studied (Table 2). Behavior change was also assessed using four statements targeting decreasing negative interactions and increasing positive interactions, positive bonds, and satisfaction or well-being. A retrospective pre-then-post survey instrument design was intentionally used as a good fit for the BYTK programming in order to evaluate learning outcomes both before and after the program for several reasons (see Marshall, Higginbotham, XXXX, and Lee, 2007) summarized below. It is from this survey thath participants were selected for this research. The experimental pretest-posttest design using a control or comparison group is considered to be one of the most respected methods that can be used to measure change in individuals (Campbell & Stanley, 1966; Kaplan, 2004). This design is highly regarded because of its control over internal validity and reliability concerns and ability to compare results from the same people or groups of people at multiple time points. This is because of the utilization of a pre-then-post survey instrument design that eliminated any illegible responses and tests or tests and responses that were irrelevant to the study. Nonetheless, this research did not entail a comparison or control group in its design that was implemented as a retrospective pretest followed by a posttest design.
While there are advantages to using the pretest-posttest method, there are some limitations with this research method as well. One limitation comes with finding an adequate comparison group, which can be difficult or impossible for the researchers to locate. Another limitation concerns the possible lack of resources and time available for community-based programs to complete comprehensive pretest-post-test comparisons (Brooks & Gersh, 1998). Also, in order for the pretest-posttest comparisons to be meaningful, participants must attend the complete program from start to finish (Pratt, McGuigan, & Katzey, 2000). Due to the nature of community education programs, attrition and sporadic attendance are common issues (Pratt, McGuigan, & Katzev, 2000).
While the pretest-posttest information must be complete for comparisons to be made, it may be challenging for researchers to see the actual changes in attitudes, behaviors, or skills if the participants overstate their original attitudes, behaviors, or skills when completing the pretest (Howard & Daily, 1979). This overestimation may occur when the participants do not have a clear understanding of the attitudes, behaviors, or skills that the program is targeting (Pratt, McGuigan, & Katzev, 2000). A lack of knowledge on certain topics (e.g., attitudes, behaviors, skills) often supports the initial need for a program intervention, but this same issue may show participants during the program that they actually knew much less than they thought when they completed the pretest. Thus, one must be aware of the potentially misleading information from pretest-posttest comparisons due to the participants’ change in perspective (Howard & Daily, 1979). “Response shift bias,” first referred to by Howard and Daily (1979), explains the “program-produced change in the participants’ understanding of the construct being measured” (Pratt, McGuigan, & Katzev, 2000, p. 342). Response shift bias, along with the issues noted previously, should be examined when reviewing findings from pretest-posttest comparisons.
Effect size. The data were analyzed using descriptive statistics and paired sample t tests. Effect sizes were calculated in order to evaluate the standardized mean differences before and after the program for each variable being studied. Focusing on effect size rather than statistical significance helps researchers determine the magnitude of standardized mean differences for a given sample and for specific identified variables. Cohen (1988) loosely characterized effect sizes as small (d = >.20), medium (d = >.50), and large (d = >.80). Further, Cohen identified a small effect size as a meaningful mean difference, a medium effect size as noticeable mean difference, and a large effect size as a clearly evident mean difference (Howell, 2002). Because it is difficult to separate program pedagogy from content, the authors of the current study determined that using effect size to evaluate standardized mean differences from before and after the BYTK program implementation was a viable first step to exploring and assessing the effectiveness of the AIAI-FTFD teaching model in facilitating change in an educational learning environment.
Results-before You Tie the Knot: Mapping Pedagogy, Learning Outcomes, and Effect Size
Results of the implementation, and evaluation of the Before You Tie the Knot program using the AIAI-FTFD teaching model as an intervention indicated clearly evident reported standardized mean changes specific to each variable being studied (Table 2). The data collected was analyzed through performance of correlation and regression analysis so as to determine the standardized mean changes. Clearly evident (large) standardized mean changes were reported by participants in their understanding of how to meet their own and their partner’s needs, parent effectively, resolve conflict in healthy ways, avoid using negative communication strategies while using positive communication strategies, manage money well, and practice healthy lifestyles. Overall, a large, clearly evident effect size (d = 1.48) was reported by participants for perceived knowledge gain from before to after participation in the BYTK program.
Results of BYTK Retrospective Pretest to Posttest Change: Before and After Programming (n=93)
|Retrospective Pretest Mean Score||Post-test Mean Score||Mean Change||Cohen’s d|
|Knowledge Change||(SD)||(SD)||(SD Pooled)||t||p||(Effect Size)|
|I understand how to meet my own 8 Needs.||3.39||4.57||1.18||12.09||.000***||1.23|
|I understand how to help my partner meet his/her 8 Needs.||3.03||4.35||1.32||12.31||.000***||1.25|
|I understand how to parent effectively.||2.85||4.28||1.43||14.41||.000***||1.47|
|I understand how to resolve conflict in constructive ways.||3.39||4.53||1.14||11.78||.000***||1.21|
|I understand how to avoid using negative communication strategies (e.g., 4 Don’ts).||3.16||4.49||1.34||12.95||.000***||1.33|
|I understand how to use positive communication strategies (e.g., 5 Do’s).||3.44||4.53||1.08||11.65||.000***||1.19|
|I understand how to manage money well.||3.06||4.21||1.15||10.96||.000***||1.13|
|I understand how to practice healthy living strategies.||3.60||4.38||0.78||8.65||.000***||0.89|
|Overall, I understand how I could use the BYTK Skills in my marriage/ relationships.||2.89||4.38||1.48||14.49||.000***||1.48|
|I meet my own 8 Needs successfully.||3.06||4.17||1.11||11.91||.000***||1.22|
|I help my partner meet his/her 8 Needs successfully.||3.17||4.03||0.86||9.67||.000***||0.99|
|I practice the principles associated with parenting effectively, even though I may not have a child yet.||3.14||4.17||1.03||11.42||.000***||1.17|
|I resolve conflicts in constructive ways.||3.21||4.26||1.05||10.56||.000***||1.08|
|I avoid using negative communication strategies (e.g., 4 Don’ts).||3.01||4.28||1.27||12.23||.000***||1.25|
|I use positive communication strategies (e.g., 5 Do’s).||3.44||4.38||0.94||10.32||.000***||1.06|
|I manage money well.||3.04||4.12||1.07||10.33||.000***||1.06|
|I practice healthy living strategies.||3.45||4.21||0.76||8.25||.000***||0.84|
|Overall, I am confident in my ability to use the BYTK Skills successfully in my marriage/relationships.||2.83||4.23||1.40||13.59||.000***||1.40|
|Overall, I use healthy skills to increase positive interactions in my premarital relationship.||3.30||4.17||0.87||10.16||.000***||1.05|
|Overall, I use healthy skills to decrease negative interaction in my premarital relationship.||3.35||4.17||0.82||9.58||.000***||0.99|
|Overall, I used healthy skills to increase positive bonds (friendship) in my premarital relationship.||3.38||4.26||0.87||10.16||.000***||1.05|
|Overall, I used healthy skills to increase happiness and satisfaction in my premarital relationship.||3.36||4.23||0.87||10.45||.000***||1.06|
* p < .05; ** p < .01; *** p < .001 Effect Size Change (d): .20=small; .50=medium; .80 or higher=large
Clearly evident standardized mean changes were also reported by participants in their confidence and behavior change from before to after the BYTK program for all of the variables studied. Overall, participants’ reported large, clearly evident standardized mean change associated with confidence gain in their ability to use the BYTK skills successfully in their marriage/relationships (d = 1.40). Perhaps most importantly, reported behavioral data revealed effect size scores ranging from d = .99 to d = 1.06 for decreasing negative interaction and increasing positive bonds, interaction, and well-being. Percentage comparisons of improvement for each variable studied from before to after BYTK programing are listed in Table 3. It is interesting to note the dynamic between participants’ overall understanding and overall confidence in using the BYTK skills successfully in relationships. It seems clear from the data that it is generally easier to understand a needed relationship skill, but it is more difficult to gain confidence in one’s ability to implement the skill successfully.
Comparison of Improvements from Before to After BYTK Programming (n=93)
Meeting personal needs successfully, managing money well, resolving conflicts in constructive ways, avoiding using negative communication strategies, and parenting effectively were all behavioral skills reported by participants in this study that revealed over 30% improvement from before to after the BYTK programming intervention was administered. Surprisingly, practicing and understanding healthy living strategies showed the lowest levels of improvement from before to after programming, although it should be remembered that participants reported large, clearly evident standardized mean change improvement for both understanding and practicing healthy living strategies.
Discussion-before You Tie the Knot: Mapping Pedagogy, Learning Outcomes, and Effect Size
Exploring the magnitude of the cognitive, emotional, and behavioral learning outcome changes associated with employing the AIAI-FTFD teaching model as an intervention in designing, delivering, and evaluating the Before You Tie the Knot (BYTK) program in an online learning environment was the purpose of this study. Because it is difficult in an educational learning environment to implement true experimental or quasi-experimental designs, using a retrospective pre-test then post-test design is a practical option for program evaluation given the inevitable challenges with this type of program (Marshall, Higginbotham, XXXX, & Lee 2007).
The AIAI-FTFD teaching model was designed to facilitate the process of change in an instructional setting (XXXX et al., 2014; Mace, 1981). Because the model is designed to facilitate change in the teaching of any content in any context, its theoretical foundation assumes that a majority of the measured change is due to the effective use of the model and not to the specific content or the context (XXXX et al., 2014). It appears from the data in this study that the AIAI-FTFD teaching model may be a viable instructional method for facilitating meaningful, noticeable, and clearly evident cognitive, emotional, and behavior change (Cohen, 1988). However, the authors readily acknowledge that content and context do exert an influence on learning outcomes but suggest that without engaging instructional design and delivery this influence can be substantially weakened (Reiser & Dempsey, 2012; Vygotsky, 1978). The analysis of cognitive (i.e., knowledge) standardized mean differences from before to after the BYTK program indicated that participants generally reported large, clearly evident increases in their understanding of how to meet their own and their partner’s needs, parent effectively, resolve conflict in healthy ways, avoid using negative communication strategies, manage money well, and practice healthy lifestyles. Additionally, participants reported large, clearly evident gains in their ability to apply (i.e., confidence) this knowledge to practice the skills introduced throughout the BYTK program successfully. Similarly, participants’ reported overall confidence in their ability to use this knowledge revealed large, clearly evident effects, as a result of the BYTK program. The reported cognitive and emotional (i.e., confidence) gains also resulted in participants’ clearly evident overall behavior change with regard to increasing positive interaction, positive bonds, and happiness/satisfaction (well-being) and decreasing negative interaction in their relationships.
The AIAI-FTFD teaching model requires instructors to identify cognitive, emotional, and behavioral target skills prior to teaching, to operationalize them into objectives, and then to map them throughout the teaching preparation and delivery process in order to maximize participant learning outcomes. Providing participants with an opportunity to practice the cognitive, emotional, and behavioral target skills within the learning environment and a way to continue to practice them through homework or using a tracking chart outside of the learning environment is one way the AIAI-FTFD teaching model assists instructors to facilitate meaningful change and maximize potential learning outcomes. The application of this model is shown specifically through the results described above regarding the BYTK curriculum and training.
Marshall (XXXX, 2010) has identified ignorance (lack of appropriate knowledge), incompetence (lack of appropriate skills), and resistance to conscience (an unwillingness to use appropriate knowledge and skills) as three primary impediments to change. As a result, instructors intentional targeting of knowledge, application, and skills throughout the learning process is key to increasing positive learning outcomes. While many models of learning and instruction target knowledge, application, and skills as important learning outcomes, few, offer a specific methodology to design, implement, and evaluate these outcomes in an easy-to-learn and start-to-finish way for educators. Educators across multiple disciplines who have used and mastered the AIAI-FTFD teaching methodology have reported meaningful qualitative and quantitative gains in their teaching effectiveness and in learner outcomes (XXXX et al., 2014; XXXX, XXXX, & Schmeer, 2016). The current study offers evidence to the existing and growing body of literature that the AIAI-FTFD teaching model may be effective in facilitating change in an educational learning environment among human services educators and clients even though there are plausible alternative explanations to the contrary.
Limitations and Implications-before You Tie the Knot: Mapping Pedagogy, Learning Outcomes, and Effect Size
Limitations of this study include the one time cross-sectional design, as well as, weaknesses of retrospective surveys as put forward by More and Tananis and Nimon, Zigarami and Allen (More and Tananis, 2009; Nimon, Zigarami and Allen, 2011). It was not possible to assess how robust the self-reported changes in knowledge, confidence, and intent to change behavior were given the study’s design. A cross-sectional evaluation and then three-month follow-up evaluation of the BYTK curriculum will need to be conducted with another sample.
Additionally, the theoretical foundation of the model was developed for the teaching of any content in any context, so it assumes that a majority of the measured change is due to the effective use of the model and not to the specific content or the context (XXXX et al., 2014). Due to the design of the study and absence of a comparison group, it cannot be determined how significantly the content and context of the BYTK training may have influenced the outcomes reported related to the teaching model. The authors noted these factors may affect learning outcomes and that further study is needed using longitudinal and comparative designs.
Another limitation of this study is the self-report nature of the survey instrument. Self-report can provide both advantages and disadvantages in conducting research. Advantages include the ease and lack of expense associated with conducting research as well as the ability to assess individual perceptions about certain constructs and variables. Disadvantages include multiple cognitive and situational internal validity issues such as history, selection, and response bias. Additionally, external validity issues also exist. Therefore, the results in this study, as with most exploratory studies, must be interpreted with caution.
The implication of this study is that meeting personal needs successfully, managing money well, resolving conflicts in constructive ways, avoiding using negative communication strategies, and parenting effectively are behavioral skills that must be addressed in the AIAI-FTFD teaching model for it to be effective in premarital education.
This study represents an ongoing attempt to explore how the AIAI-FTFD teaching model can be used to facilitate change in instructional and programmatic settings. Results of this study indicate that the AIAI-FTFD teaching model did not inhibit but indeed may have facilitated change in cognitive, emotional, and behavioral learning outcomes among participants in this study. These initial results offer potential future directions for study of the AIAI-FTFD model, including longitudinal evaluations and follow up studies across different contexts and subject areas. Using the model to design, implement, and evaluate premarital education programming represents another tool in the tool box educators can use to intentionally pursue effective instruction and programming in an educational learning environment.
Allgood, S., Higgenbotham, B., & Skogrand, L. (2007a). Creating rituals in stepfamilies. Logan, UT: Utah State University Extension.
Allgood, S., Higgenbotham, B., & Skogrand, L. (2007b). Helpful strategies to deal with ex-partners in remarriages. Logan, UT: Utah State University Extension.
Amato, P.R., Johnson, D.R., Booth, A., & Rogers, S.L. (2003). Continuity and change in marital quality between 1980 and 2000. Journal of Marriage and Family, 65(1), 1-22. doi: 10.1111/j.1741-3737.2003.00001.x
Anderson, L.W., & Krathwohl, D.R. (Eds.). Airasian, P.W., Cruikshank, K.A., Mayer, R.E., Pintrich, P.R., Raths, J., & Wittrock, M.C. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s Taxonomy of Educational Objectives (Complete edition). New York: Longman.
Badger, R. I. (Ed.). (2008). Ideas that work in college teaching. Albany, NY: State University of New York Press.
Bennett, C., & Rockwell, K. (1995). Targeting outcomes of programs (TOP): An integrated approach to planning and evaluation. Unpublished manuscript, University of Nebraska, Lincoln.
Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook 1: Cognitive domain. New York: David McKay.
Brooks, L., & Gersh, T. L. (1998). Assessing the impact of diversity initiatives using the retrospective pretest design. Journal of College Student Development, 34, 383-385.
Bubolz, M.M., & Sontag, S.M. (1993). Human ecology theory. In P.G. Boss, W.J. Doherty, R. LaRossa, W.R. Schumm, & S.K. Steinmetz (Eds.), Sourcebook of family theories and methods: A contextual approach. New York: Plenum.
Campbell, D. T., & Stanley, J. C. (1966). Experimental and quasi-experimental designs for research. Chicago: Rand McNally.
Cline, F., & Fay, J. (2006). Parenting with love and logic. Colorado Springs, CO: NavPress.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York: Academic.
Cole, J. M. (1981, September/October). Selecting Extension teaching methods. Journal of Extension, 27-32. Retrieved from http://www.joe.org/joe/1981september/81-5-a4.pdf
Coplen, R.D., & MacArthur, J.D. (1982). Developing a healthy self-image. Provo, UT: Brigham Young University Press.
DeNoon, D. J. (2003). Marriage satisfaction key to women’s health benefits. Retrieved from http://www.health.harvard.edu/newsletter_article/marriage-and-mens-health
Dew, J. (2008). Debt change and marital satisfaction change in recently married couples. Family Relations, 57, 60–71. doi:10.1111/j.1741-3729.2007.00483.x
Doss, B. D., Rhoades, G. K., Stanley, S. M., Markman, H.J., & Johnson, C. A. (2009). Differential use of premarital education in first and second marriages. Journal of Family Psychology, 23, 268 – 273. doi:10.1037/a0014356
Edgar, D. (1969). Audio-visual methods in teaching (3rd ed.). New York: Holt, Rinehart & Winston.
Fawcett, E. B., Hawkins, A. J., Blnchard, V. L., & Carroll, J. S. (2010). Do Premarital Education Programs Really Work? A Meta‐analytic Study. Family Relations, 59(3), 232-239. doi: 10.1111/j.1741-3729.2010.00598.x
Gagné, R. M., Briggs, L. J., & Wager, W. W. (1992). Principles of instructional design (4th ed.). Forth Worth, TX: Harcourt Brace Jovanovich College Publishers.
Gottman, J. M. (1994a). Why marriages succeed or fail. New York: Fireside.
Gottman, J. M. (1994b). What predicts divorce? The relationship between marital process and marital outcomes. Hillsdale, NJ: Erlbaum.
Gottman, J. M., Coan, J., Carrere, S., & Swanson, C. (1998). Predicting marital happiness and stability from newlywed interactions. Journal of Marriage and Family, 60, 5-22. doi:10.2307/353438
Gottman, J. M. and Notarius, C. I. (2000). Decade Review: Observing Marital Interaction. Journal of Marriage and Family, 62, 927–947. doi:10.1111/j.1741-3737.2000.00927.x
Hackathorn, J., Solomon, E. D., Blankmeyer, K. L., Tennial, R. E., & Garczynski, A. M. (2011). Learning by doing: An empirical study of active teaching techniques. The Journal of Effective Teaching, 11(2), 40-54. Retrieved from http://uncw.edu/cte/et/articles/Vol11_2/Hackathorn.pdf
Harrison, C. (2011). Premarital preparation requirements in state law. National Healthy Marriage Resource Center. Retrieved from http://www.healthymarriageinfo.org/resource-detail/index.aspx?rid=3687
Harvard Medical School. (2010). Marriage and men’s health. Harvard Men’s Health Watch. Boston: Harvard Health Publications. Retrieved from http://www.health.harvard.edu/newsletters/Harvard_Mens_Health_Watch/2010/July/marriage-and-mens-health
Hitti, M. (2007). Get married, gain weight. Retrieved from http://www.webmd.com/diet/news/20071024/get-married-gain-weight
Howard, G. S., & Daily, P. R. (1979). Response-shift bias: A source of contamination of self-report measures. Journal of Applied Psychology, 64, 144-150. doi:10.1037/0021-9010.64.2.144
Howell, D.C. (2002). Statistical methods for psychology (5th ed.). Pacific Grove, CA: Wadsworth.
Kaplan, D. (Ed.). (2004). The Sage handbook of quantitative methodology for the social sciences. Thousand Oaks, CA: Sage.
Larson, J. H. (2003). The great marriage tune-up book. San Francisco: Jossey-Bass.
Larson, J. H., & Holman, T. B. (1994). Predictors of marital quality and stability. Family Relations, 43, 228-237. doi:10.2307/585327
Latham, G. I. (2002). Behind the schoolhouse door: Eight skills every instructor should have. In G.I. Latham, Behind the schoolhouse door: Managing chaos with science, skills, and strategy (pp. 11-41). North Logan, UT: P & T Ink.
Latham, G. I. (1994). The power of positive parenting. North Logan, UT: P & T Ink.
Mace, D. (1981). The long trail from information giving to behavioral change. Family Relations, 30, 599-606. doi:10.2307/584350
Markey, C.N., Markey, P.M., & Gray, H.F. (2007). Romantic relationships and health: An examination of individuals’ perceptions of their romantic partners’ influences on their health. Sex Roles, 57, 435-445. doi:10.1007/s11199-007-9266-5
Marshall, J. P., Higginbotham, B., XXXX, V.W., & Lee, T.R. (2007). Assessing program outcomes: Rationale and benefits of posttest-then-retrospective-pretest designs. Journal of Youth Development: Bridging Research and Practice, 2(1), Article 0701RS001. doi:http://dx.doi.org/10.5195/jyd.2007.366
Merrill, M. D. (1991). Constructivism and instructional design. Educational Technology, 31(5), 45-53.
Merrill, M. D. (1997). First principles of instruction: A synthesis. In R.A. Reiser & J.V. Dempsey (Eds.), Trends and issues in instructional design and technology (2nd Ed. pp. 62-71). Upper Saddle River, NJ: Prentice Hall.
Michel, N., Cater, J. J., & Varela, O. (2009). Active versus passive teaching styles: An empirical study of student learning outcomes. Human Resource Development Quarterly, 20(4), 397-418. doi:10.1002/hrdq.20025
Moore & Tananis. (2009). Measuring change in a short term educational program using a retrospective pretest design. American Journal of Evaluation, 30, 189-202.
Nimon, Zigarami, & Allen (2011). Measures of Program Effectiveness Based on Retrospective Pretest Data: Are All Created Equal? American Journal of Evaluation, 32, 8-28.
Online Sunshine: Official Internet Site of the Florida Legislature. (2016). The 2016 Florida Statutes: Chapter 741: Marriage and Domestic Violence. Retrieved from http://www.leg.state.fl.us/Statutes/index.cfm?App_mode=Display_Statute&URL=0 700-0799/0741/0741.html
Paas, F., Renkl, A., and Sweller, J. (2004). Cognitive load theory: Instructional implications of the interaction between information structures and cognitive architecture. Instructional Science, 32, 1-8. doi:10.1023/B:TRUC.0000021806.17516.d0
Powell, H., & Cassidy, D. (2007). Family life education: Working with families across the lifespan. Long Grove, IL: Waveland Press.
Pratt, C. C., McGuigan, W. M., & Katzev, A. R. (2000). Measuring program outcomes: Using retrospective pretest methodology. American Journal of Evaluation, 21, 341-349. doi:10.1016/S1098-2140(00)00089-8
Reiser, R. A., & Dempsey, J. V. (2012). Trends and issues in instructional design and technology (3rd Ed.). Boston, MA: Allyn & Bacon.
Rogge, R. D., Cobb, R. J., Lawrence, E., Johnson, M. D., & Bradbury, T. N. (2013). Is skills training necessary for the primary prevention of marital distress and dissolution? A 3-year experimental study of three interventions. Journal of Consulting and Clinical Psychology, 81(6), 949. doi:http://dx.doi.org/10.1037/a0034209
Roggman, L.A., Boyce, L.K., & Innocenti, M.S. (2008). Developmental parenting: A guide for early childhood practitioners. Baltimore, MA: Paul H. Brookes.
Rosenshine, B. (1983). Teaching functions in instructional programs. The Elementary School Journal, 83(4), 335-351. doi:10.1086/461321
Schramm, D.G., Marshall, J.P., XXXX, V.W., & Lee, T.R. (2005). After “I do”: The newlywed transition. Marriage and Family Review, 38(1), 45-67. doi: http://dx.doi.org/10.1300/J002v38n01_05
SMARTcouples.org. (2016). Before You Tie the Knot. Retrieved from http://smartcouples.ifas.ufl.edu/classesevents-/class-description/before-you-tie-the-knot/
Stanley, S. M., Amato, P. R., Johnson, C. A., & Markman, H. J. (2006). Premarital education, marital quality, and marital stability: Findings from a large, random household survey. Journal of Family Psychology, 20, 117-126. doi:http://dx.doi.org/10.1037/0893-3188.8.131.52
Stevenson, C. D., & Harris, G. K. (2014). Instruments for characterizing instructors’ teaching practices: A review. NACTA, 58(1-4), 104-110.
Teemant, B., Moen, D., XXXX. (2013). Problem-based learning in the family sciences: A good fit in theory and practice. Family Science Review, 17(2), 102-117. Retrieved from http://familyscienceassociation.org/sites/default/files/7%20-Teemont_Moen_XXXX.pdf
The, N.S., & Gordon-Larsen, P. (2009). Entry into romantic partnership is associated with obesity. Obesity, 17, 1441-1447. doi:10.1038/oby.2009.97
Tyler, R. W. (1949). Basic principles of curriculum and instruction (3rd ed.). Chicago: University of Chicago Press.
Vygotsky,L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University.
Wallerstein, J.S. (1996). The psychological tasks of marriage: II. American Journal of Orthopsychiatry, 66, 217-227. doi:10.1037/h0080173
Wiggins, G., & McTighe, J. (2005). Understanding by design (2nd Expanded Edition). Alexandria, VA: ASCD.