Remember what you did so you know what to do next

We explore using a moderately sized large language model (GPT-J 6B parameters) to create a plan for a simulated robot to achieve 30 classes of goals in ScienceWorld, a text game simulator for elementary science experiments. Previously published empirical work claimed that large language models (LLMs) are a poor fit (Wang et al., 2022) compared to reinforcement learning. Using the Markov assumption (a single previous step), the LLM outperforms the reinforcement learning-based approach by a factor of 1.4. When we fill the LLM's input buffer with as many prior steps as possible, improvement rises to 3.5x. Even when training on only 6.5% of the training data, we observe a 2.2x improvement over the reinforcement-learning-based approach. Our experiments show that performance varies widely across the 30 classes of actions, indicating that averaging over tasks can hide significant performance issues. In work contemporaneous with ours, Lin et al. (2023) demonstrated a two-part approach (SwiftSage) that uses a small LLM (T5-large) complemented by OpenAI's massive LLMs to achieve outstanding results in ScienceWorld. Our 6-B parameter, single-stage GPT-J matches the performance of SwiftSage's two-stage architecture when it incorporates GPT-3.5 turbo which has 29-times more parameters than GPT-J.Comment: Identical to EMNLP 2023 Finding

University–industry collaboration: using meta-rules to overcome barriers to knowledge transfer

This is the final version of the article. Available from Springer Verlag via the DOI in this record.University–industry knowledge transfer is an important source wealth of creation for all partners; however, the practical management of this activity within universities is often hampered by procedural rigidity either through the absence of decision-making protocols to reconcile conflicting priorities or through the inconsistent implementation of existing policies. This is problematic, since it can impede operational effectiveness, prevent inter-organisational knowledge-creation and hamper organisational learning. This paper addresses this issue by adopting a cross-discipline approach and presenting meta-rules as a solution to aid organisational decision making. It is proposed that meta-rules can help resolve tensions arising from conflicting priorities between academics, knowledge transfer offices and industry and help facilitate strategic alignment of processes and policies within and between organisations. This research contributes to the growing debate on the strategic challenges of managing knowledge transfer and presents meta-rules as a practical solution to facilitate strategic alignment of internal and external stakeholder tensions. Meta-rules has previously only been applied in a computer intelligence context however, this research proves the efficacy of meta rules in a university–industry knowledge transfer context. This research also has practical implications for knowledge transfer office managers who can use meta-rules to help overcome resource limitations, conflicting priorities and goals of diverse internal and external stakeholders