Manuscript Title: Before You Tie the Knot

Category:

Description

Manuscript Title: Before You Tie the Knot:

Mapping Pedagogy, Learning Outcomes, and Effect Size in Continuing Education

 

Reviewer Name:

Instructions

 

Please type specific comments on the next page that will be clear, direct, and useful to the author(s).  Identify strengths and weaknesses and provide concrete suggestions for enhancing the manuscript.  Questions are provided to help guide the feedback if you would like to use them.  Because this form applies for the manuscript categories of Original Research (traditional-style manuscripts as well as creative works), Brief Reports, Practice and Pedagogy, Theory, and Emerging Scholarship, some of the specific questions may not be applicable.

 

Then complete the checklist (for Editorial Board use only) and provide your determination regarding publication.  Include any comments for the Editor only.

 

When finished, please return this document to the Associate Editor.  He/she will then forward all comments to the Editor.

 

Manuscript Title: Before You Tie the Knot:

 

 

Thank you in advance for your constructive feedback and your service to the Journal of Human Sciences and Extension!

 

 

 

 

 

Journal of Human Sciences and Extension

Feedback for Author

 

Manuscript Title: Before You Tie the Knot:Mapping Pedagogy, Learning Outcomes, and Effect Size in Continuing Education-manuscript Title: Before You Tie the Knot:

 

Originality:  Does the manuscript advance knowledge in the field?  Is it novel and interesting to warrant publication?

The manuscript does advance knowledge in the field. The model on which the study is based, the AIAI-FTFD  Start-to-Finish Teaching Model, is a teaching tool that has been previously used in other educational settings. In this case, the manuscript presents the outcome of its use in premarital preparation training, which is both novel and warrant publication.

Structure:  Is the manuscript clearly laid out?  Can you follow the logic from the introduction to the conclusion?

The manuscript is clearly laid out, with a logical progression of introduction, background (from general to specific), purpose, methods, results, discussion, limitations and implications, and conclusions.

Title and Abstract:  Do the title and abstract reflect the manuscript as a whole as well as the findings in particular?  Are the most salient keywords included with the abstract?

Both the title and abstract reflect the manuscript as a whole. While the abstract reflects the findings as a whole, the title could better reflect the findings by including that there were large effect size gains. The keywords are included with the abstract.

Introduction:  Overall, how well has the stage been set for what is being reported?  Are relevant research and theories summarized?  Are important works missing?  How was the creative work conceived?  Was a theme constructed for the creative work?  Are research questions and/or hypotheses delineated?  Do the research questions and/or hypotheses stem from the research and theories presented?

The introduction does an adequate job of setting the stage for what is being reported in the manuscript. The author cites the value of using active learning because it has been found to produce better student gains in class specific outcomes. The reader is then informed that the AIAI-FTFD Start-to-Finish Teaching model was applied to premarital preparation training and was evaluated through the participants’ reported training outcomes.

Relevant research is summarized in the background section of the manuscript. Not only does the author provide research and theory across a time span of several decades, he/she also provides an outline of the research and theories addressed, which includes a review of the AIAI-FTFD model, its theoretical underpinnings, the curriculum to which it is applied, as well a reference for previous research that is related to the AIAI-FTFD model. There does not appear to be any important works missing.

The creative work was conceived because evidence shows that premarital education and training improve couple relationship quality and satisfaction, but that there is a need for improvement in the area, which might be met through the use of the AIAI-FTFD model to assist educators in how they deliver information and skills training.

A theme was constructed, being the model’s effectiveness in teaching the Before You Tie the Knot premarital preparation program. The purpose of the study, which stems from the research and theories presented, was to evaluate the potential of a premarital training program as an effective AIAI-FTFD model for teaching in human services educational programming. This purpose was based on the research question: “What are the cognitive, emotional, and behavioral change learning outcomes generated by employing the AIAI-FTFD teaching model as an intervention in designing, delivering, and evaluating the Before You Tie the Knot (BYTK) program in an online learning environment?” which also stems from the research and theories presented.Manuscript Title: Before You Tie the Knot:

Method:  Are the research strategies/design processes clearly described?  Is there a justification for the methods used?  Is the design suitable for answering the research questions?  Is enough detail presented to allow for replication?  Was the sampling appropriate?  Are constructs properly operationalized?  Are reliability and validity information included for measures used when appropriate?

The author clearly described the research strategy and design process, which was an expansion of the author’s previous studies, which were referenced in the methods section. The design of the study, a self-report quantitative exploratory cross-sectional design using a purposive sampling method, was specified and described in detail.

The author(s) discuss the strengths of the experimental retrospective pre-post with comparison/control group design but this is not the design and method that was employed for this study (as stated in the limitations and implications section, “Due to the design of the study and absence of a comparison group…”). This is misleading to the reader and should be removed. Instead, the design should be discussed as it was implemented as a retrospective pretest then posttest design. I have included a couple articles here that describe this design that may be useful to cite. Particularly, in discussing the limitations of this design (see comments later).  Manuscript Title: Before You Tie the Knot:

Moore & Tananis. (2009). Measuring change in a short term educational program using a retrospective pretest design. American Journal of Evaluation, 30, 189-202.

Nimon, Zigarami, & Allen (2011). Measures of Program Effectiveness Based on Retrospective Pretest Data: Are All Created Equal? American Journal of Evaluation, 32, 8-28.

Although the justification for the methods was well supported, the author did not do an adequate job of providing the details of the method. For example, while the source of the sample was appropriate because it used participants who completed the educational program, it was unclear if they were selected at random or if all participants completed the survey. Moreover, the demographics of the group could have a significant influence on the results, so this would also have to be addressed if the study was to be repeated.

Constructs were properly operationalized as the study used a Likert scale for responses and also calculated effect sizes after completing a statistical analysis. Reliability and validity information was not included.

Results:  Are the results laid out clearly and logically?  Have appropriate statistical analyses been conducted if relevant?  Are interpretations correct?  Are tables and figures used appropriately (to present data that are too complex to describe in the text – to improve readability)?  Are tables and figures clear and succinct?

The results are laid out clearly and logically, progressing from standardized mean changes specific to each variable being studied and effect size to percentage comparisons of improvement for each variable studied before and after the intervention. However, even though statistical analyses are presented, the author does not describe how the analyses were conducted. Regardless, the interpretations appear to be correct, in that the author found large effect size for confidence gained and improving positive bond, interaction, and well-being. The table related to the results, which are Table 2 and Table 3, are clear and succinct, while adequately presenting the data too complex to describe complete in the text.Manuscript Title: Before You Tie the Knot:

Discussion/Conclusions:  Are the major findings summarized concisely and accurately?  Are the conclusions supported by the results?  Has a connection been made to their expectations and previous research cited in the manuscript?  Are plausible alternative explanations for findings presented?  Are limitations of the study discussed?  Is there an explanation of how the study advanced knowledge in the field?

The major findings were summarized concisely and accurately, in that the AIAI-FTFD model was seen as a possible viable instructional model that can be used to produce meaningful cognitive, emotional, and behavioral changes. The author connected the conclusions to the before and after results of the Before Tying the Knot program standardized mean differences  and the gains and confidence levels and positive  outcomes, as reported by the participants. The connection met their expectations because they identified that the results were due to the programs curriculum and training.

, the limitations of the study are discussed. The authors did explain that the study has advanced knowledge in the field, if interpreted with caution, because it indicated that the teaching model facilitated change in the participants’ learning outcomes. The author(s) make the claim that this evaluation can provide support for the effectiveness of the teaching approach (AIAI-FTFD). However, I question the confidence in which the author(s) assert this claim as there is not sufficient evidence provided to support this claim. Had the BTK program been implemented using the AIAI-FTFD teaching approach and without utilizing this approach and comparison of the two approaches had shown statistically significant results that would have been more convincing. Without such a design, the authors should only tentatively state that this may provide support for this teaching delivery method and indicate that research of this nature would need to be conducted (i.e. research implications).

While the author(s) do identify the cross-sectional design as a limitation, they do not identify the use of a retrospective pre-post survey as a limitation. While there is literature supporting this approach, there is also plenty of literature that identifies the weaknesses of retrospective surveys. I recommend that the authors use the two citations listed above to address the limitations of retrospective surveys.

Implications:  Does the manuscript give adequate attention to practical implications of the work?  Does the manuscript fit within the scope of the journal?Manuscript Title: Before You Tie the Knot:

Although the author(s) have a limitations and implications section, they do not actually identify any practical implications of this work. While it may seem that practical implications are implied/obvious when reporting evaluation findings of an educational program, the authors still need to explicitly identify practical implications of this work. The nature of the content reported on in this manuscript does fit within the scope of the journal but only if practical implications are identified. Writing:  Is the writing clear?  Has the use of jargon been avoided?  Is the writing concise?  Are grammar and punctuation correct?  Are there typographical errors?

The writing is clear, concise, and avoids the use of jargon. No specific typographical errors were noted, but there are some minor grammatical errors. For example, in the second paragraph of the introduction, line 6, the word “is” has been omitted.

 

Manuscript Title: Before You Tie the Knot:

Mapping Pedagogy, Learning Outcomes, and Effect Size in Continuing Education

Rating Checklist

  Yes Marginal No N/A
Originality:   
Manuscript advances knowledge in the field x      
Structure:   
Manuscript clearly laid out x      
Title and Abstract:  
Reflect manuscript   x    
Introduction:  
Relevant research and theories summarized x      
Research questions and/or hypotheses delineated x      
Methods:  
Research strategies clearly described and justified   X    
Sampling appropriate x      
Constructs operationalized        
Detail provided to allow for replication     x  
Results:  
Appropriate statistical analyses conducted   x    
Interpretations of analyses correct x      
Tables and figures used appropriately x      
Discussion/Conclusions:  
Major findings summarized accurately x      
Conclusions supported by results   X    
Connection made to expectations and previous research x      
Limitations of study discussed x      
Implications:  
Adequate attention given to practical implications of the work     X  
Manuscript fits within scope of journal x      
Writing:  
Writing clear and concise x      
Minimal grammar, punctuation, and/or typographical errors x      

 

 

Publication Recommendation  
Accept in present form  
Accept contingent on minor revisions  
Revise and resubmit (major revisions needed) X
Reject  

 

Please provide any feedback for the Editor only below.

Overall, this is an important study that was well conducted and provided meaningful results for the human service sector and specifically for pre-marital programs. The manuscript needs a few minor revisions, in addition to a few grammatical errors it contains. Firstly, the title could be revised to reflect a statement about the findings. The methods section needs a more complete description of how the participants were selected from the pool of those taking the on-line course and an explanation of how the statistical analysis was conducted.