Effectiveness of Teaching Metamorphic Testing

This paper is an attempt to understand the effectiveness of teaching metamorphic properties in a senior/graduate software engineering course classroom environment through gauging the success achieved by students in identifying these properties on the basis of the lectures and materials provided in class. The main findings were: (1) most of the students either misunderstood what metamorphic properties are or fell short of identifying all the metamorphic properties in their respective projects, (2) most of the students that were successful in finding all the metamorphic properties in their respective projects had incorporated certain arithmetic rules into their project logic, and (3) most of the properties identified were numerical metamorphic properties. A possible reason for this could be that the two relevant lectures given in class cited examples of metamorphic properties that were based on numerical properties. Based on the findings of the case study, pertinent suggestions were made in order to improve the impact of lectures provided for Metamorphic Testing

HELP ME THINK: A Simple Prompting Strategy for Non-experts to Create Customized Content with Models

Controlling the text generated by language models and customizing the content has been a long-standing challenge. Existing prompting techniques proposed in pursuit of providing control are task-specific and lack generality; this provides overwhelming choices for non-expert users to find a suitable method for their task. The effort associated with those techniques, such as in writing examples, explanations, instructions, etc. further limits their adoption among non-expert users. In this paper, we propose a simple prompting strategy HELP ME THINK where we encourage GPT3 to help non-expert users by asking a set of relevant questions and leveraging user answers to execute the task. We demonstrate the efficacy of our technique HELP ME THINK on a variety of tasks. Specifically, we focus on tasks that are hard for average humans and require significant thinking to perform. We hope our work will encourage the development of unconventional ways to harness the power of large language models.Comment: ACL 2023 Finding

Investigating the Failure Modes of the AUC metric and Exploring Alternatives for Evaluating Systems in Safety Critical Applications

With the increasing importance of safety requirements associated with the use of black box models, evaluation of selective answering capability of models has been critical. Area under the curve (AUC) is used as a metric for this purpose. We find limitations in AUC; e.g., a model having higher AUC is not always better in performing selective answering. We propose three alternate metrics that fix the identified limitations. On experimenting with ten models, our results using the new metrics show that newer and larger pre-trained models do not necessarily show better performance in selective answering. We hope our insights will help develop better models tailored for safety-critical applications

Structure−Function Correlation of Chloroquine and Analogues as Transgene Expression Enhancers in Nonviral Gene Delivery

To understand how chloroquine (CQ) enhances transgene expression in polycation-based, nonviral gene delivery systems, a number of CQ analogues with variations in the aliphatic amino side chain or in the aromatic ring are synthesized and investigated. Our studies indicate that the aliphatic amino moiety of CQ is essential to provide increased gene expression. Further, the enhancements are more dramatically affected by changes to the aromatic ring and are positively correlated to the strength of intercalation between DNA and the CQ analogues. Quinacrine (QC), a CQ analogue with a fused acridinyl structure that can strongly intercalate DNA, enhances transfection similarly to CQ at a concentration 10 times lower, while N^4-(4-pyridinyl)-N^1,N^1-diethyl-1,4-pentanediamine (CP), a CQ analogue that has a weakly intercalating pyridinyl ring, shows no effect on gene expression. Subtle change on the 7-substituent of the chloroquine aromatic structure can also greatly affect the ability of the CQ analogues to enhance transgene expression. Transfection in the presence of N^4-(7-trifluoromethyl-4-quinolinyl)-N^1,N^1-diethyl-1,4-pentanediamin e (CQ7a) shows expression efficiency 10 times higher than in the presence of CQ at same concentration, while transfection in the presence of N^4-(4-quinolinyl)-N^1,N^1-diethyl-1,4-pentanediamine (CQ7b) does not reveal any enhancing effects on expression. Through a number of comparative studies with CQ and its analogues, we conclude that there are at least three mechanistic features of CQ that lead to the enhancement in gene expression:  (i) pH buffering in endocytic vesicles, (ii) displacement of polycations from the nucleic acids in polyplexes, and (iii) alteration of the biophysical properties of the released nucleic acid