Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1
The low sense of belonging and self-efficacy have been identified as key factors for underreprese... more The low sense of belonging and self-efficacy have been identified as key factors for underrepresented students not choosing computing careers and retaining in computing disciplines. Researchers advised that without adequate mentorship and role models, many of these students do not view computing as a viable option. Even though several works utilized mentoring, they do not provide detailed guidelines on how this may have been accomplished and how it could be applied to other settings. In this study, we explore the efficacy of incorporating mentoring into the curriculum of an introductory undergraduate computing course. In this work, we seek the answer to the following two research questions: (1) How do culturally diverse mentor-mentee relationships impact the sense of belonging, computing identity, and self-efficacy of underrepresented students in computing programs? (2) How does the integration of mentoring initiatives influence the perceptions of underrepresented students toward computing majors? We implemented mentoring practices in the Fall of 2022 and results show that our mentoring interventions were able to improve participants' sense of belonging and computing identity. Mentors and mentees also shared positive opinions toward our initiatives. CCS CONCEPTS • Applied computing → Interactive learning environments; Collaborative learning.
The following Research-to-Practice full paper presents the outcomes of a remote synchronous flipp... more The following Research-to-Practice full paper presents the outcomes of a remote synchronous flipped classroom implementation of a senior-level data management course. As flipped classroom models of instruction have gained popularity in higher education, it has prompted the need for an investigation into content-specific methods for flipped curriculum design. Our work identified and implemented flipped classroom design factors in an Advanced Database Design (CIT 44400) course to address a gap in research around flipped classroom models within undergraduate data-science courses. Through literature review, we identified a set of eight factors of effective flipped classroom instruction and evaluated them via survey, focus group, course evaluation, student performance, and interview data. We segmented designed course activities which incorporated these eight factors with a set of learning objectives that emphasized collaborative iterative practices to position students as designers and evaluators of solutions to industry-authentic problems. Instructional design factors were evaluated via course evaluations and student surveys followed by an activity-based qualitative analysis of focus group, instructor interview, and student free-response data to understand student perceptions of instructional approaches, intended versus practical outcomes of such activities, and guidelines for future course design iterations and research. We argue instructional design must be student-centered and consider student goals alongside those that are ‘scripted’ into the course structure to best serve, motivate, and engage students. During the Fall 2020 implementation of CIT 44400, we prioritized learner independence, peer collaboration, and critical thinking in the instructional design, selecting flipped methodologies with the intention of fostering these skills in senior students. In light of Covid-19, the course was adapted to be synchronous online. Regardless of these unforeseen constraints, student performance and course evaluation data indicate that with peer and instructor feedback, students were able to apply course content appropriately in their final independent project as evidenced by a 12% increase in assessment scores and an improvement in students average overall final course grades from a ‘B’ to an ‘A-’ as compared to the previously taught Fall 2019 lecture-based section of the same class. Course evaluation scores also improved from 3.22 to 3.66 from Fall 2019 to Fall 2020. Self-reported survey data from students indicate that (a) feedback from the instructor, (b) small group work, (c) revision of work based on feedback, and (d) solution evaluations were the most positively impactful instructional design elements throughout the course. In particular, students mentioned one-on-one scaffolding from the instructor as beneficial for their learning. Students also communicated challenges faced during the pandemic, complaints, and recommendations for future course iterations. Data from focus group discussions conveyed that students (a) generally had positive learning experiences despite the constraints of learning exclusively online and (b) developed the industry-specific collaborative practices which were designed into the forefront of our instructional model. The flipped classroom design and implementation process in this research is transformative and can be employed by other STEM disciplines to design a domain-specific flipped model for their classroom which considers the needs of one's students, the challenges of the current time, and the state of the field at large.
With the rapid growth of the Social Web, a large amount of informal opinionated texts are availab... more With the rapid growth of the Social Web, a large amount of informal opinionated texts are available on numerous topics. Natural language tools for automatically analyzing these opinions become necessary to help individuals, organizations, and governments in making timely decisions. A query-based opinion summarizer from opinionated documents can ad-I would like to thank Dr. Leila Kosseim, my academic adviser, for all her guidance, support, and enthusiasm. I particularly appreciated that despite her enormous workload she always made herself available to me and took the time to revise my thesis and papers for publications. I would like to express my deep appreciation to the members of my doctoral committee: Professors Gregory Butler, Sabine Bergler, Nizar Bouguila, and Guy Lapalme for their valuable feedback on my thesis. Their comments, questions and suggestions have been very useful to improve my Ph.D. work. I would like to thank my husband, Quamrul Islam Khan, for his endless love, support, and care during all these years. I would like to thank my parents for their moral support and inspiration which have brought me here today.
2019 IEEE Frontiers in Education Conference (FIE), 2019
This research to practice full paper describes our cross-course knowledge integration approach, w... more This research to practice full paper describes our cross-course knowledge integration approach, which uses a project-based learning environment. The structure and instructional pedagogies of some of our data management concentration courses in our department have not changed in a long time. Most of our data management courses cover theoretical knowledge without showing their practical applications. As a result, students don’t find those courses interesting enough. In addition, none of our data management courses require cross-course interactions, which is the current instructional pedagogical direction. With the goal of addressing these issues and improving student learning, faculty employed a cross-course knowledge integration model to teach the advanced database design course CIT 44400. We redesigned this advanced database course as part of our data management curriculum enhancement also to align with the trends of the IT industry and to enable students to correlate knowledge lear...
The author has granted a nonexclusive license alloîiving Library and Archives Canada to reproduce... more The author has granted a nonexclusive license alloîiving Library and Archives Canada to reproduce, publish, archive, preserye, conserve, communicate to the public by telecommunication or on the lnternet loan, distribute and selltheses worldwide, for commerc¡al or noncommercial purposes, in microform, paper, electronic andlor any other formats. The author retains copyright ownership and moral rights in this thesis. Neither the thesis nor substantial extracts from it may be printed or otherwise reproduced without the author's permission. 0-494-08918-0 AVIS: L'auteur a accordé une licence non exclusive permettant à la Bibliothèque et Archives Canada de reproduire, publier, archiver, sauvegarder; conserver, transmettre au public par télécommunication ou par I'lnternet, prêter, distribuer et vendre des thèses partout dans le monde, à des fins commerciales ou autres, sur support microforme, papier, électronique et/ou autres formats. Your file /SA/Vi Our file /SBN; Votre référence Notre retërence ln compliance with the Canadian Privacy Act some supporting forms may have been removed from this thesis. Whiie these forms may be included in the document page count, their removal does not represent any loss of content from the thesis. f,*¡ L'auteur conserve la propriété du droit d'auteur et des droits moraux qui protège cette thèse. Ni la thèse ni des extraits substantiels de celle-ci ne doivent être imprimés ou autrement reproduits sans son autorisation. Canada Conformément à fa loi canadienne sur la protection de la vie pr¡vée, quelques formulaires secondaires ont été enlevés de cette thèse. Bien que ces formulaires aient inclus dans la pagination, il n'y aura aucun contenu manquant. THE IINWERSITY OF MANITOBA FACULTY OF GRADUATE STT]DIES ***rF* Yerbaluation in the Context of a Tutoring System for Description Logics by COPYRIGHT PERMISSION A ThesisAEracticum submitted to the Faculty of Graduate Studies of The University of Manitoba in partial fulfillment of the requirement of the degree of Master of Science Shamima Mithun Permission has been granted to the Library of the University of Manitoba to lend or sell copies of this thesis/practicum, to the National Library of Canada to microfilm this thesis and to lend or sell copies of the film, and to University Microfilms Inc. to publish an abstract of this thesis/practicum. This reproduction or copy of this thesis has been made available by authority of the copyright owner solely for the purpose of private study and research, andmay oniy be reproduced and copied as permitted by copyright laws or with express written authorization from the copyright owner. Shamima Mithun O 2005 provides a language called Powerloom, which is a DL dialect. For my M. Sc. thesis, I developed a Verbalization Enhanced Tutoring System (VETS), applied to the knowledge representation language provided by the Powerloom system.
As more and more people are expressing their opinions on the web in the form of weblogs (or blogs... more As more and more people are expressing their opinions on the web in the form of weblogs (or blogs), research on the blogosphere is gaining popularity. As the outcome of this research, different natural language tools such as query-based opinion summarizers have been developed to mine and organize opinions on a particular event or entity in blog entries. However, the variety of blog posts and the informal style and structure of blog entries pose many difficulties for these natural language tools. In this paper, we identify and categorize errors which typically occur in opinion summarization from blog entries and compare blog entry summaries with traditional news text summaries based on these error types to quantify the differences between these two genres of texts for the purpose of summarization. For evaluation, we used summaries from participating systems of the TAC 2008 opinion summarization track and updated summarization track. Our results show that some errors are much more fre...
Intelligent Tutoring Systems (ITS) typically contain elements of instruction, assessment, feedbac... more Intelligent Tutoring Systems (ITS) typically contain elements of instruction, assessment, feedback and guidance for the trainee. Most of the time, the ITS is controlling the dialogue with the learner, whereas the learner has a passive, reactive role. In this paper, we suggest a technique called "verbalization", which allows users to work independently in the domain of application, and still get useful feedback and support. The principle of verbalization is to generate a natural language expression reflecting the meaning of a user's input. We developed and tested this concept in VETS, a Verbalization Enhanced Tutoring System for PowerLoom, a knowledge representation language based on description logic. Our studies showed that verbalization improves the persistence of users in trying to accomplish their goals and consequently leads to higher success rates in solving given tasks.
With the rapid growth of the Social Web, a large amount of informal opinionated texts are availab... more With the rapid growth of the Social Web, a large amount of informal opinionated texts are available on numerous topics. However, people can be overwhelmed with this vast amount of information and they need help to find the information of their interests. Natural language tools for automatically analyzing these opinions become necessary to help individuals, organizations, and governments in making timely decisions. To address this need, I proposed a summarization approach for opinionated texts. To validate my approach, BlogSum is developed and evaluated experimentally using current benchmarks. Users can ask BlogSum any question (e.g. Why do people like Chrome better than Firefox?). To answer user's question, BlogSum first retrieves relevant blogs, reviews from the web then generates a concise summary that represents people opinions expressed towards the topic.
Discourse incoherence is an important and typical problem with multi-document extractive summarie... more Discourse incoherence is an important and typical problem with multi-document extractive summaries. To address this issue, we have developed a schema-based summarization approach for query-based blog summaries that utilizes discourse structures. In our schema design, we tried to model discourse structures which are typically used by humans in their summary writing in response to a particular question type. In our approach, a sentence instantiates a specific slot of the schema based on its discourse structures. To validate our approach, we have built a system named BlogSum and have evaluated its performance through 4 human participants using a likert scale of 1 to 5. The evaluation results show that our approach has significantly improved summary coherence compared to the summaries with no discourse structuring without compromising on content evaluation.
Today, automatic evaluation metrics such as ROUGE have become the de-facto mode of evaluating an ... more Today, automatic evaluation metrics such as ROUGE have become the de-facto mode of evaluating an automatic summarization system. However, based on the DUC and the TAC evaluation results, (Conroy and Schlesinger, 2008; Dang and Owczarzak, 2008) showed that the performance gap between human-generated summaries and system-generated summaries is clearly visible in manual evaluations but is often not reflected in automated evaluations using ROUGE scores. In this paper, we present our own experiments in comparing the results of manual evaluations versus automatic evaluations using our own text summarizer: BlogSum. We have evaluated BlogSum-generated summary content using ROUGE and compared the results with the original candidate list (OList). The t-test results showed that there is no significant difference between BlogSum-generated summaries and OList summaries. However, two manual evaluations for content using two different datasets show that BlogSum performed significantly better than ...
In this paper, we propose a domain and genre-independent approach to identify the discourse relat... more In this paper, we propose a domain and genre-independent approach to identify the discourse relation called attributive, included in Grimes’ relation list [7]. An attributive relation provides details about an entity or an event or can be used to illustrate a particular feature about a concept or an entity. Since attributive relations describe attributes or features of an object or an event, they are often used in text summarization (e.g. [2]) and question answering systems (e.g. [12]). However, to our knowledge, no previous work has focused on tagging attributive relations automatically. We propose an automatic domain and genre-independent approach to tag attributive relations by utilizing dependency relations of words based on dependency grammars [3]. In this paper, we also show how attributive relations can be utilized in text summarization. By using a subset of the BLOG06 corpus, we have evaluated the accuracy of our attributive classifier and compared it to a baseline and human...
The availability of huge amounts of online opinions has created a new need to develop effective q... more The availability of huge amounts of online opinions has created a new need to develop effective query-based opinion summarizers to analyze this information in order to facilitate decision making at every level. To develop an effective opinion summarization approach, we have targeted to resolve specifically Question Irrelevancy and Discourse Incoherency problems which have been found to be the most frequently occurring problems for opinion summarization. To address these problems, we have introduced a hybrid approach by combining text schema and rhetorical relations to exploit intra-sentential rhetorical relations. To evaluate our approach, we have built a system called BlogSum and have compared BlogSum-generated summaries after applying rhetorical structuring to BlogSum-generated candidate sentences without utilizing rhetorical relations using the Text Analysis Conference (TAC) 2008 data for summary contents. Evaluation results show that our approach improves summary contents by red...
The work presented in this paper attempts to evaluate and quantify the use of discourse relations... more The work presented in this paper attempts to evaluate and quantify the use of discourse relations in the context of blog summarization and compare their use to more traditional and factual texts. Specifically, we measured the usefulness of 6 discourse relations - namely comparison, contingency, illustration, attribution, topic-opinion, and attributive for the task of text summarization from blogs. We have evaluated the effect of each relation using the TAC 2008 opinion summarization dataset and compared them with the results with the DUC 2007 dataset. The results show that in both textual genres, contingency, comparison, and illustration relations provide a significant improvement on summarization content; while attribution, topic-opinion, and attributive relations do not provide a consistent and significant improvement. These results indicate that, at least for summarization, discourse relations are just as useful for informal and affective texts as for more traditional news articles.
2020 IEEE Frontiers in Education Conference (FIE), 2020
This Research to Practice Full Paper presents a framework to evaluate and design flipped classroo... more This Research to Practice Full Paper presents a framework to evaluate and design flipped classroom activities for data science and management courses. Variants of flipped classrooms have been employed in STEM fields with great success in students’ learning outcomes. Research shows that flipped classrooms would improve students’ learning if it is implemented following rigorous procedures of an efficient instructional design. As a result, one of the critical focus of current flipped classroom research is what factors educators need to consider when designing a flipped learning environment. Currently, educators incorporate various factors such as "pre-recorded video lecture", "group activity" as a trial and error basis and adjust these factors based on their own experience and students’ feedback. On the other hand, the emergence of big data expects a new graduate to demonstrate mastery of concepts and skills for data acquisition, management, and analysis of inferenc...
We would like to thank the organisers of the 7th International Conference on Recent Advances in N... more We would like to thank the organisers of the 7th International Conference on Recent Advances in Natural Language Processing, RANLP 2009, for hosting this workshop as one of their satellite events. We would also like to thank the researchers who submitted papers and the members of Programme Committee for their help in the reviewing process.
The availability of huge amounts of online opinions has created a new need to develop effective q... more The availability of huge amounts of online opinions has created a new need to develop effective query-based opinion summarizers to analyze this information in order to facilitate decision making at every level. To develop an effective opinion summarization approach, we have targeted to resolve specifically Question Irrelevancy and Discourse Incoherency problems which have been found to be the most frequently occurring problems for opinion summarization. To address these problems, we have introduced a hybrid approach by combining text schema and rhetorical relations to exploit intra-sentential rhetorical relations. To evaluate our approach, we have built a system called BlogSum and have compared BlogSum-generated summaries after applying rhetorical structuring to BlogSum-generated candidate sentences without utilizing rhetorical relations using the Text Analysis Conference (TAC) 2008 data for summary contents. Evaluation results show that our approach improves summary contents by reducing question irrelevant sentences.
Intelligent Tutoring Systems (ITS) typically contain elements of instruction, assessment, feedbac... more Intelligent Tutoring Systems (ITS) typically contain elements of instruction, assessment, feedback and guidance for the trainee. Most of the time, the ITS is controlling the dialogue with the learner, whereas the learner has a passive, reactive role. In this paper, we suggest a technique called "verbalization", which allows users to work independently in the domain of application, and still get useful feedback and support. The principle of verbalization is to generate a natural language expression reflecting the meaning of a user's input. We developed and tested this concept in VETS, a Verbalization Enhanced Tutoring System for PowerLoom, a knowledge representation language based on description logic. Our studies showed that verbalization improves the persistence of users in trying to accomplish their goals and consequently leads to higher success rates in solving given tasks.
Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1
The low sense of belonging and self-efficacy have been identified as key factors for underreprese... more The low sense of belonging and self-efficacy have been identified as key factors for underrepresented students not choosing computing careers and retaining in computing disciplines. Researchers advised that without adequate mentorship and role models, many of these students do not view computing as a viable option. Even though several works utilized mentoring, they do not provide detailed guidelines on how this may have been accomplished and how it could be applied to other settings. In this study, we explore the efficacy of incorporating mentoring into the curriculum of an introductory undergraduate computing course. In this work, we seek the answer to the following two research questions: (1) How do culturally diverse mentor-mentee relationships impact the sense of belonging, computing identity, and self-efficacy of underrepresented students in computing programs? (2) How does the integration of mentoring initiatives influence the perceptions of underrepresented students toward computing majors? We implemented mentoring practices in the Fall of 2022 and results show that our mentoring interventions were able to improve participants' sense of belonging and computing identity. Mentors and mentees also shared positive opinions toward our initiatives. CCS CONCEPTS • Applied computing → Interactive learning environments; Collaborative learning.
The following Research-to-Practice full paper presents the outcomes of a remote synchronous flipp... more The following Research-to-Practice full paper presents the outcomes of a remote synchronous flipped classroom implementation of a senior-level data management course. As flipped classroom models of instruction have gained popularity in higher education, it has prompted the need for an investigation into content-specific methods for flipped curriculum design. Our work identified and implemented flipped classroom design factors in an Advanced Database Design (CIT 44400) course to address a gap in research around flipped classroom models within undergraduate data-science courses. Through literature review, we identified a set of eight factors of effective flipped classroom instruction and evaluated them via survey, focus group, course evaluation, student performance, and interview data. We segmented designed course activities which incorporated these eight factors with a set of learning objectives that emphasized collaborative iterative practices to position students as designers and evaluators of solutions to industry-authentic problems. Instructional design factors were evaluated via course evaluations and student surveys followed by an activity-based qualitative analysis of focus group, instructor interview, and student free-response data to understand student perceptions of instructional approaches, intended versus practical outcomes of such activities, and guidelines for future course design iterations and research. We argue instructional design must be student-centered and consider student goals alongside those that are ‘scripted’ into the course structure to best serve, motivate, and engage students. During the Fall 2020 implementation of CIT 44400, we prioritized learner independence, peer collaboration, and critical thinking in the instructional design, selecting flipped methodologies with the intention of fostering these skills in senior students. In light of Covid-19, the course was adapted to be synchronous online. Regardless of these unforeseen constraints, student performance and course evaluation data indicate that with peer and instructor feedback, students were able to apply course content appropriately in their final independent project as evidenced by a 12% increase in assessment scores and an improvement in students average overall final course grades from a ‘B’ to an ‘A-’ as compared to the previously taught Fall 2019 lecture-based section of the same class. Course evaluation scores also improved from 3.22 to 3.66 from Fall 2019 to Fall 2020. Self-reported survey data from students indicate that (a) feedback from the instructor, (b) small group work, (c) revision of work based on feedback, and (d) solution evaluations were the most positively impactful instructional design elements throughout the course. In particular, students mentioned one-on-one scaffolding from the instructor as beneficial for their learning. Students also communicated challenges faced during the pandemic, complaints, and recommendations for future course iterations. Data from focus group discussions conveyed that students (a) generally had positive learning experiences despite the constraints of learning exclusively online and (b) developed the industry-specific collaborative practices which were designed into the forefront of our instructional model. The flipped classroom design and implementation process in this research is transformative and can be employed by other STEM disciplines to design a domain-specific flipped model for their classroom which considers the needs of one's students, the challenges of the current time, and the state of the field at large.
With the rapid growth of the Social Web, a large amount of informal opinionated texts are availab... more With the rapid growth of the Social Web, a large amount of informal opinionated texts are available on numerous topics. Natural language tools for automatically analyzing these opinions become necessary to help individuals, organizations, and governments in making timely decisions. A query-based opinion summarizer from opinionated documents can ad-I would like to thank Dr. Leila Kosseim, my academic adviser, for all her guidance, support, and enthusiasm. I particularly appreciated that despite her enormous workload she always made herself available to me and took the time to revise my thesis and papers for publications. I would like to express my deep appreciation to the members of my doctoral committee: Professors Gregory Butler, Sabine Bergler, Nizar Bouguila, and Guy Lapalme for their valuable feedback on my thesis. Their comments, questions and suggestions have been very useful to improve my Ph.D. work. I would like to thank my husband, Quamrul Islam Khan, for his endless love, support, and care during all these years. I would like to thank my parents for their moral support and inspiration which have brought me here today.
2019 IEEE Frontiers in Education Conference (FIE), 2019
This research to practice full paper describes our cross-course knowledge integration approach, w... more This research to practice full paper describes our cross-course knowledge integration approach, which uses a project-based learning environment. The structure and instructional pedagogies of some of our data management concentration courses in our department have not changed in a long time. Most of our data management courses cover theoretical knowledge without showing their practical applications. As a result, students don’t find those courses interesting enough. In addition, none of our data management courses require cross-course interactions, which is the current instructional pedagogical direction. With the goal of addressing these issues and improving student learning, faculty employed a cross-course knowledge integration model to teach the advanced database design course CIT 44400. We redesigned this advanced database course as part of our data management curriculum enhancement also to align with the trends of the IT industry and to enable students to correlate knowledge lear...
The author has granted a nonexclusive license alloîiving Library and Archives Canada to reproduce... more The author has granted a nonexclusive license alloîiving Library and Archives Canada to reproduce, publish, archive, preserye, conserve, communicate to the public by telecommunication or on the lnternet loan, distribute and selltheses worldwide, for commerc¡al or noncommercial purposes, in microform, paper, electronic andlor any other formats. The author retains copyright ownership and moral rights in this thesis. Neither the thesis nor substantial extracts from it may be printed or otherwise reproduced without the author's permission. 0-494-08918-0 AVIS: L'auteur a accordé une licence non exclusive permettant à la Bibliothèque et Archives Canada de reproduire, publier, archiver, sauvegarder; conserver, transmettre au public par télécommunication ou par I'lnternet, prêter, distribuer et vendre des thèses partout dans le monde, à des fins commerciales ou autres, sur support microforme, papier, électronique et/ou autres formats. Your file /SA/Vi Our file /SBN; Votre référence Notre retërence ln compliance with the Canadian Privacy Act some supporting forms may have been removed from this thesis. Whiie these forms may be included in the document page count, their removal does not represent any loss of content from the thesis. f,*¡ L'auteur conserve la propriété du droit d'auteur et des droits moraux qui protège cette thèse. Ni la thèse ni des extraits substantiels de celle-ci ne doivent être imprimés ou autrement reproduits sans son autorisation. Canada Conformément à fa loi canadienne sur la protection de la vie pr¡vée, quelques formulaires secondaires ont été enlevés de cette thèse. Bien que ces formulaires aient inclus dans la pagination, il n'y aura aucun contenu manquant. THE IINWERSITY OF MANITOBA FACULTY OF GRADUATE STT]DIES ***rF* Yerbaluation in the Context of a Tutoring System for Description Logics by COPYRIGHT PERMISSION A ThesisAEracticum submitted to the Faculty of Graduate Studies of The University of Manitoba in partial fulfillment of the requirement of the degree of Master of Science Shamima Mithun Permission has been granted to the Library of the University of Manitoba to lend or sell copies of this thesis/practicum, to the National Library of Canada to microfilm this thesis and to lend or sell copies of the film, and to University Microfilms Inc. to publish an abstract of this thesis/practicum. This reproduction or copy of this thesis has been made available by authority of the copyright owner solely for the purpose of private study and research, andmay oniy be reproduced and copied as permitted by copyright laws or with express written authorization from the copyright owner. Shamima Mithun O 2005 provides a language called Powerloom, which is a DL dialect. For my M. Sc. thesis, I developed a Verbalization Enhanced Tutoring System (VETS), applied to the knowledge representation language provided by the Powerloom system.
As more and more people are expressing their opinions on the web in the form of weblogs (or blogs... more As more and more people are expressing their opinions on the web in the form of weblogs (or blogs), research on the blogosphere is gaining popularity. As the outcome of this research, different natural language tools such as query-based opinion summarizers have been developed to mine and organize opinions on a particular event or entity in blog entries. However, the variety of blog posts and the informal style and structure of blog entries pose many difficulties for these natural language tools. In this paper, we identify and categorize errors which typically occur in opinion summarization from blog entries and compare blog entry summaries with traditional news text summaries based on these error types to quantify the differences between these two genres of texts for the purpose of summarization. For evaluation, we used summaries from participating systems of the TAC 2008 opinion summarization track and updated summarization track. Our results show that some errors are much more fre...
Intelligent Tutoring Systems (ITS) typically contain elements of instruction, assessment, feedbac... more Intelligent Tutoring Systems (ITS) typically contain elements of instruction, assessment, feedback and guidance for the trainee. Most of the time, the ITS is controlling the dialogue with the learner, whereas the learner has a passive, reactive role. In this paper, we suggest a technique called "verbalization", which allows users to work independently in the domain of application, and still get useful feedback and support. The principle of verbalization is to generate a natural language expression reflecting the meaning of a user's input. We developed and tested this concept in VETS, a Verbalization Enhanced Tutoring System for PowerLoom, a knowledge representation language based on description logic. Our studies showed that verbalization improves the persistence of users in trying to accomplish their goals and consequently leads to higher success rates in solving given tasks.
With the rapid growth of the Social Web, a large amount of informal opinionated texts are availab... more With the rapid growth of the Social Web, a large amount of informal opinionated texts are available on numerous topics. However, people can be overwhelmed with this vast amount of information and they need help to find the information of their interests. Natural language tools for automatically analyzing these opinions become necessary to help individuals, organizations, and governments in making timely decisions. To address this need, I proposed a summarization approach for opinionated texts. To validate my approach, BlogSum is developed and evaluated experimentally using current benchmarks. Users can ask BlogSum any question (e.g. Why do people like Chrome better than Firefox?). To answer user's question, BlogSum first retrieves relevant blogs, reviews from the web then generates a concise summary that represents people opinions expressed towards the topic.
Discourse incoherence is an important and typical problem with multi-document extractive summarie... more Discourse incoherence is an important and typical problem with multi-document extractive summaries. To address this issue, we have developed a schema-based summarization approach for query-based blog summaries that utilizes discourse structures. In our schema design, we tried to model discourse structures which are typically used by humans in their summary writing in response to a particular question type. In our approach, a sentence instantiates a specific slot of the schema based on its discourse structures. To validate our approach, we have built a system named BlogSum and have evaluated its performance through 4 human participants using a likert scale of 1 to 5. The evaluation results show that our approach has significantly improved summary coherence compared to the summaries with no discourse structuring without compromising on content evaluation.
Today, automatic evaluation metrics such as ROUGE have become the de-facto mode of evaluating an ... more Today, automatic evaluation metrics such as ROUGE have become the de-facto mode of evaluating an automatic summarization system. However, based on the DUC and the TAC evaluation results, (Conroy and Schlesinger, 2008; Dang and Owczarzak, 2008) showed that the performance gap between human-generated summaries and system-generated summaries is clearly visible in manual evaluations but is often not reflected in automated evaluations using ROUGE scores. In this paper, we present our own experiments in comparing the results of manual evaluations versus automatic evaluations using our own text summarizer: BlogSum. We have evaluated BlogSum-generated summary content using ROUGE and compared the results with the original candidate list (OList). The t-test results showed that there is no significant difference between BlogSum-generated summaries and OList summaries. However, two manual evaluations for content using two different datasets show that BlogSum performed significantly better than ...
In this paper, we propose a domain and genre-independent approach to identify the discourse relat... more In this paper, we propose a domain and genre-independent approach to identify the discourse relation called attributive, included in Grimes’ relation list [7]. An attributive relation provides details about an entity or an event or can be used to illustrate a particular feature about a concept or an entity. Since attributive relations describe attributes or features of an object or an event, they are often used in text summarization (e.g. [2]) and question answering systems (e.g. [12]). However, to our knowledge, no previous work has focused on tagging attributive relations automatically. We propose an automatic domain and genre-independent approach to tag attributive relations by utilizing dependency relations of words based on dependency grammars [3]. In this paper, we also show how attributive relations can be utilized in text summarization. By using a subset of the BLOG06 corpus, we have evaluated the accuracy of our attributive classifier and compared it to a baseline and human...
The availability of huge amounts of online opinions has created a new need to develop effective q... more The availability of huge amounts of online opinions has created a new need to develop effective query-based opinion summarizers to analyze this information in order to facilitate decision making at every level. To develop an effective opinion summarization approach, we have targeted to resolve specifically Question Irrelevancy and Discourse Incoherency problems which have been found to be the most frequently occurring problems for opinion summarization. To address these problems, we have introduced a hybrid approach by combining text schema and rhetorical relations to exploit intra-sentential rhetorical relations. To evaluate our approach, we have built a system called BlogSum and have compared BlogSum-generated summaries after applying rhetorical structuring to BlogSum-generated candidate sentences without utilizing rhetorical relations using the Text Analysis Conference (TAC) 2008 data for summary contents. Evaluation results show that our approach improves summary contents by red...
The work presented in this paper attempts to evaluate and quantify the use of discourse relations... more The work presented in this paper attempts to evaluate and quantify the use of discourse relations in the context of blog summarization and compare their use to more traditional and factual texts. Specifically, we measured the usefulness of 6 discourse relations - namely comparison, contingency, illustration, attribution, topic-opinion, and attributive for the task of text summarization from blogs. We have evaluated the effect of each relation using the TAC 2008 opinion summarization dataset and compared them with the results with the DUC 2007 dataset. The results show that in both textual genres, contingency, comparison, and illustration relations provide a significant improvement on summarization content; while attribution, topic-opinion, and attributive relations do not provide a consistent and significant improvement. These results indicate that, at least for summarization, discourse relations are just as useful for informal and affective texts as for more traditional news articles.
2020 IEEE Frontiers in Education Conference (FIE), 2020
This Research to Practice Full Paper presents a framework to evaluate and design flipped classroo... more This Research to Practice Full Paper presents a framework to evaluate and design flipped classroom activities for data science and management courses. Variants of flipped classrooms have been employed in STEM fields with great success in students’ learning outcomes. Research shows that flipped classrooms would improve students’ learning if it is implemented following rigorous procedures of an efficient instructional design. As a result, one of the critical focus of current flipped classroom research is what factors educators need to consider when designing a flipped learning environment. Currently, educators incorporate various factors such as "pre-recorded video lecture", "group activity" as a trial and error basis and adjust these factors based on their own experience and students’ feedback. On the other hand, the emergence of big data expects a new graduate to demonstrate mastery of concepts and skills for data acquisition, management, and analysis of inferenc...
We would like to thank the organisers of the 7th International Conference on Recent Advances in N... more We would like to thank the organisers of the 7th International Conference on Recent Advances in Natural Language Processing, RANLP 2009, for hosting this workshop as one of their satellite events. We would also like to thank the researchers who submitted papers and the members of Programme Committee for their help in the reviewing process.
The availability of huge amounts of online opinions has created a new need to develop effective q... more The availability of huge amounts of online opinions has created a new need to develop effective query-based opinion summarizers to analyze this information in order to facilitate decision making at every level. To develop an effective opinion summarization approach, we have targeted to resolve specifically Question Irrelevancy and Discourse Incoherency problems which have been found to be the most frequently occurring problems for opinion summarization. To address these problems, we have introduced a hybrid approach by combining text schema and rhetorical relations to exploit intra-sentential rhetorical relations. To evaluate our approach, we have built a system called BlogSum and have compared BlogSum-generated summaries after applying rhetorical structuring to BlogSum-generated candidate sentences without utilizing rhetorical relations using the Text Analysis Conference (TAC) 2008 data for summary contents. Evaluation results show that our approach improves summary contents by reducing question irrelevant sentences.
Intelligent Tutoring Systems (ITS) typically contain elements of instruction, assessment, feedbac... more Intelligent Tutoring Systems (ITS) typically contain elements of instruction, assessment, feedback and guidance for the trainee. Most of the time, the ITS is controlling the dialogue with the learner, whereas the learner has a passive, reactive role. In this paper, we suggest a technique called "verbalization", which allows users to work independently in the domain of application, and still get useful feedback and support. The principle of verbalization is to generate a natural language expression reflecting the meaning of a user's input. We developed and tested this concept in VETS, a Verbalization Enhanced Tutoring System for PowerLoom, a knowledge representation language based on description logic. Our studies showed that verbalization improves the persistence of users in trying to accomplish their goals and consequently leads to higher success rates in solving given tasks.
Uploads
Papers by Shamima Mithun