Interacciones entre Procalus (Coleoptera, Chrysomelidae) y Lithraea caustica (Sapindales, Anacardiaceae). Un caso de monofagia en el matorral de Chile central

Experiments and observations have been perfomed in order to show the effective dependence of Procalus (Chrysomelidae) beetles on Lithraea caustica (Anacardiaceae) shrubs in a preandean matorral site in Central Chile. Procalus insects, particularely P. viridis, are only found on L. caustica, even when they are not able to recognize their host plant while flying. This fact could be explained by the obsewation that beetles abandon L. caustica shrubs more slowly than other shrub species. This behavior of adults is coherent with lamae constraints because the latter feed only on L. caustica new leaves in spite of their previous alimentary experience. The above results support the idea that Procalus beetles are monophagous of L. caustica in the study site. Finally, as the phenology of L. caustica in the study site is highly unpredictable both intra and interannually, a possible phenological coupling mechanism is proposed to explain this monophagous relationship.En este trabajo se reafizan experimentos y observaciones con el proposito de mostrar el grado de dependencia de los coleópteros del género Procalus (Chrysomelidae) sobre el arbusto Lithraea caustica (Anacardiaceae) en un sector de matorral preandino en Chile central. Los resultados indican que los insectos del género Procalus, en particular P. viridis, no son capaces de reconocer su planta hospedadora en vuelo, aunque se distribuyen solo sobre L. caustica. Esto podria tener su explicación en el hecho de que los insectos abandonan los arbustos de L. caustica más lentamente de lo que lo hacen con otras especies arbustivas. Esta conducta de los adultos es coherente con las limitaciones biológicas de las lamas, pues éstas, tengan o no experiencia alimentaria previa, sólo se alimentan de hojas nuevas de L. caustica. Se muestra asila monofagia de Procalus por L. caustica en el lugar de estudio. Sin embargo, como la fenologia de L. caustica en el mismo lugar es altamente impredecible de año en año e incluso dentro del atio, se propone un posible mecanismo de acopiamiento fenológico entre los coleópteros del género Procalus y L. caustica

Novel perspectives on multi-fidelity sequential decision-making

DOTTORATONegli ultimi anni, i problemi di decisione sequenziali hanno ricevuto molta attenzione nel campo dell'intelligenza artificiale, in quanto modellano un ampio spettro di problemi significativi. In questi scenari, un agente interagisce nel tempo con un ambiente al fine di raggiungere un certo scopo, e il concetto chiave è che ogni decisione dell'agente influenzerà le opzioni future e i risultati. In questo contesto, diverse tecniche generali sono state sviluppate negl corso degli anni per risolvere varie sfide. Per esempio, queste metodologie includono algoritmi multi-armed bandit, oltre a metodi di reinforcement learning e inverse reinforcement learning. Nonostante queste tecniche abbiamo mostrato risultati promettenti, e talvolta sorprendenti, richiedono spesso un numero significativamente alto di interazioni con l'ambiente per raggiungere risultati soddisfanceti. Tuttavia, in molte applicazioni reali, è possibile raccogliere e sfruttare dati imprecisi ma meno costosi (ad esempio, interazioni generate interagendo con un modello a bassa fedeltà dell'ambiente) al fine di rendere il processo di addestramento più efficiente. Per questa ragione, algoritmi di multi-fidelity learning hanno recentemente attirato l'attenzione come una strada promettente per bilanciare le prestazioni e l'efficienza dell'apprendimento. In questa dissertazione, esploreremo problemi legati all'apprendimento multi-fidelity da diverse prospettive innovative. Il nostro obiettivo primario è migliorare l'efficienza dell'apprendimento in problemi di decisione sequenziale, sfruttando dati meno costosi, ma anche meno informativi, che sono disponibili in diversi scenari. I nostri contributi, in tal senso, possono essere suddivisi in tre parti. Nella prima parte, studiamo una nuova variante multi-fidelity del problema best-arm identification. Nel problema classico di best-arm identification, l'agente ha accesso a un insieme di braccia, ciascuna associata a un segnale di ricompensa, e deve identificare il braccio migliore minimizzando il numero di interazioni con l'ambiente. Nella variante multi-fidelity, invece, l'agente ha anche accesso a un insieme di approssimazioni meno costose per ciascun braccio, che può sfruttare per ridurre il costo di identificazione. Nella seconda parte, studiamo come migliorare le tecniche di stima Monte Carlo per i problemi di decisione sequenziale sfruttando la funzione di reset comunemente disponibile in molti simulatori utilizzati per reinforcement learning. Più nello specifico, abbiamo identificato e cerchiamo di sfruttare una struttura multi-fidelity nascosta e intrinseca all'interno degli algoritmi Monte Carlo di reinforcement learning, ovvero la capacità di un agente di interrompere una traiettoria Monte Carlo per ottenere un'interazione meno informativa, ma meno costosa, con l'ambiente. Infine, nella terza parte, investigiamo il problema di inverse reinforcement learning, in cui l'obiettivo è dedurre una funzione di rinforzo che sia in grado di spiegare il comportamento di un agente esperto che opera in un ambiente di decisione sequenziale. In questa dissertazione, consideriamo una variante multi-fidelity in cui il sistema di apprendimento può anche osservare il comportamento di molteplici esperti sub-ottimali. Infatti, raccogliere dimostrazioni di comportamenti sub-ottimali è di solito meno costoso rispetto all'osservazione della politica di un agente ottimale.Over the last years, sequential decision-making problems have received significant attention in AI, as they model a broad spectrum of real-world problems. In these problems, an agent interacts with an environment over time to achieve a specific goal, and the key peculiarity is that each decision the agent makes will influence future options and outcomes. In this context, a large variety of general techniques have been developed over the years to address various challenges. For example, these methodologies include multi-armed bandit algorithms, as well as reinforcement learning and inverse reinforcement learning methods. Although these techniques have shown promising, and even surprising, results, they often require a large number of interactions with the environment to achieve a satisfactory performance level. However, in several real-world applications, imprecise but cheaper data (e.g., interactions generated by using a low-fidelity model of the environment) can be collected and exploited to make the training process more efficient. In this context, multi-fidelity learning algorithms have recently gained attention as a promising option for balancing performance and learning efficiency. In this dissertation, we explore multi-fidelity learning from several novel perspectives. Our primary goal is to enhance learning efficiency in sequential decision-making problems by understanding how to leverage cheaper, yet less informative data, which commonly appear in various scenarios. The contributions can thus be broadly divided into three parts. In the first part, we study a novel multi-fidelity variant of the best-arm identification problem. In the classical best-arm identification problem, the agent has access to a set of arms, each associated with a reward signal, and it needs to identify the optimal arm while minimizing the number of interactions with the environment. In the multi-fidelity variant, instead, the agent has access to a set of cheaper approximations for each arm, that can exploited to reduce the identification cost. In the second part, we study how to improve Monte Carlo estimation techniques for sequential decision-making problems by leveraging the reset function commonly available in many reinforcement learning simulators. Specifically, we identify and seek to exploit a hidden, intrinsic multi-fidelity structure within Monte Carlo Reinforcement Learning algorithms, that is the ability of an agent to truncate a Monte Carlo trajectory to obtain a less informative, but cheaper, interaction with the environment. Finally, in the third part, we investigate the inverse reinforcement learning problem, where the goal is to infer a reward function that explains the behavior of an expert agent acting in a sequential decision-making environment. In this dissertation, we consider a multi-fidelity variant in which the learning system can also observe the behavior of multiple sub-optimal experts. Indeed, gathering demonstrations of sub-optimal behavior is usually cheaper than observing the policy of an optimal agent.DIPARTIMENTO DI ELETTRONICA, INFORMAZIONE E BIOINGEGNERIAComputer Science and Engineering37GATTI, NICOLAPIRODDI, LUIG

Sperm storage by males causes changes in sperm phenotype and influences the reproductive fitness of males and their sons

Recent studies suggest that environmentally induced effects on sperm phenotype can influence offspring phenotype beyond the classic Mendelian inheritance mechanism. However, establishing whether such effects are conveyed purely through ejaculates, independently of maternal environmental effects, remains a significant challenge. Here, we assess whether environmentally induced effects on sperm phenotype affects male reproductive success and offspring fitness. We experimentally manipulated the duration of sperm storage by males, and thus sperm age, in the internally fertilizing fish Poecilia reticulata. We first confirm that sperm ageing influences sperm quality and consequently males reproductive success. Specifically, we show that aged sperm exhibit impaired velocity and are competitively inferior to fresh sperm when ejaculates compete to fertilize eggs. We then used homospermic (noncompetitive) artificial insemination to inseminate females with old or fresh sperm and found that male offspring arising from fertilizations by experimentally aged sperm suffered consistently impaired sperm quality when just sexually mature (four months old) and subsequently as adults (13 months old). Although we have yet to determine whether these effects have a genetic or epigenetic basis, our analyses provide evidence that environmentally induced variation in sperm phenotype constitutes an important source of variation in male reproductive fitness that has far reaching implications for offspring fitness

Ocurrence of Nyssomyia intermedia (Lutz & Neiva) (Diptera: Psychodidae) in Paraná State, south of Brazil

Nyssomyia intermedia s. lat. tem sido citada por vários autores no Paraná. No entanto, alguns estudos apontam que esse táxon corresponde a Nyssomyia neivai (Pinto). Em coletas realizadas em galinheiro e em ambiente de mata, com armadilhas, entre novembro de 2005 e outubro de 2006, em Adrianópolis, Morretes e Pontal do Paraná, localizados na região de Mata Atlântica na Serra do Mar e no litoral do Paraná, sete fêmeas de Nyssomyia intermedia s. str. (Lutz & Neiva) foram encontradas juntamente com outras 14 espécies de flo ebotomíneos, confirmando a ocorrência de N.intermedia em área de costa e de mata Atlântica do Paraná.The occurrence of Nyssomyia intermedia s.lat. in the state of Paraná, Brazil, has been registered by several authors; however, studies have identified this taxon as belonging, in Paraná, to Nyssomyia neivai (Pinto). During captures with traps in a hen-house and forested areas, from November 2005 to October 2006, in Adrianópolis, Morretes and Pontal do Paraná, situated in the Atlantic forest domain, Paraná state, seven females of Nyssomyia intermedia s.str. (Lutz & Neiva) were collected together with other 14 sand floy species. Thus the occurrence of N. intermedia on the coast and in areas of Atlantic forest in Paraná is confirmed

Sequential Transfer in Reinforcement Learning with a Generative Model

We are interested in how to design reinforcement learning agents that provably reduce the sample complexity for learning new tasks by transferring knowledge from previously-solved ones. The availability of solutions to related problems poses a fundamental trade-off: whether to seek policies that are expected to achieve high (yet sub-optimal) performance in the new task immediately or whether to seek information to quickly identify an optimal solution, potentially at the cost of poor initial behavior. In this work, we focus on the second objective when the agent has access to a generative model of state-action pairs. First, given a set of solved tasks containing an approximation of the target one, we design an algorithm that quickly identifies an accurate solution by seeking the state-action pairs that are most informative for this purpose. We derive PAC bounds on its sample complexity which clearly demonstrate the benefits of using this kind of prior knowledge. Then, we show how to learn these approximate tasks sequentially by reducing our transfer setting to a hidden Markov model and employing spectral methods to recover its parameters. Finally, we empirically verify our theoretical findings in simple simulated domains.Comment: ICML 202

Pure Exploration under Mediators' Feedback

Stochastic multi-armed bandits are a sequential-decision-making framework, where, at each interaction step, the learner selects an arm and observes a stochastic reward. Within the context of best-arm identification (BAI) problems, the goal of the agent lies in finding the optimal arm, i.e., the one with highest expected reward, as accurately and efficiently as possible. Nevertheless, the sequential interaction protocol of classical BAI problems, where the agent has complete control over the arm being pulled at each round, does not effectively model several decision-making problems of interest (e.g., off-policy learning, partially controllable environments, and human feedback). For this reason, in this work, we propose a novel strict generalization of the classical BAI problem that we refer to as best-arm identification under mediators' feedback (BAI-MF). More specifically, we consider the scenario in which the learner has access to a set of mediators, each of which selects the arms on the agent's behalf according to a stochastic and possibly unknown policy. The mediator, then, communicates back to the agent the pulled arm together with the observed reward. In this setting, the agent's goal lies in sequentially choosing which mediator to query to identify with high probability the optimal arm while minimizing the identification time, i.e., the sample complexity. To this end, we first derive and analyze a statistical lower bound on the sample complexity specific to our general mediator feedback scenario. Then, we propose a sequential decision-making strategy for discovering the best arm under the assumption that the mediators' policies are known to the learner. As our theory verifies, this algorithm matches the lower bound both almost surely and in expectation. Finally, we extend these results to cases where the mediators' policies are unknown to the learner obtaining comparable results

Truncating Trajectories in Monte Carlo Reinforcement Learning

In Reinforcement Learning (RL), an agent acts in an unknown environment to maximize the expected cumulative discounted sum of an external reward signal, i.e., the expected return. In practice, in many tasks of interest, such as policy optimization, the agent usually spends its interaction budget by collecting episodes of fixed length within a simulator (i.e., Monte Carlo simulation). However, given the discounted nature of the RL objective, this data collection strategy might not be the best option. Indeed, the rewards taken in early simulation steps weigh exponentially more than future rewards. Taking a cue from this intuition, in this paper, we design an a-priori budget allocation strategy that leads to the collection of trajectories of different lengths, i.e., truncated. The proposed approach provably minimizes the width of the confidence intervals around the empirical estimates of the expected return of a policy. After discussing the theoretical properties of our method, we make use of our trajectory truncation mechanism to extend Policy Optimization via Importance Sampling (POIS, Metelli et al., 2018) algorithm. Finally, we conduct a numerical comparison between our algorithm and POIS: the results are consistent with our theory and show that an appropriate truncation of the trajectories can succeed in improving performance

Sistemas de Spins Interagentes: uma abordagem pedagógica

Teaching Physics is inherently challenging. However, with the growing popularity of quantum mechanics, the topic is sometimes associated with non-scientific contexts, which urges the dissemination of basic knowledge about physics of the quantum realm in elementary education. Therefore, this study aims to promote an accessible and uncomplicated introduction to interacting spin systems, using the Heisenberg model to analyze spin chains. To this end, we built a theoretical foundation that supports the diagonalization of matrices and the study of quantum phenomena, including addressing the Kondo Effect. Furthermore, from the creation and use of computer programs, we obtained exact numerical results that allowed us to interpret relevant physical quantities, which were fundamental for the contextualization of interacting systems. All of this is contextualized for basic education students using tangible and colorful fitting pieces to illustrate concepts from the quantum world. This approach fits into the gamification methodology, which makes learning more engaging and accessible to students. Students become protagonists in the construction of knowledge, taking advantage of the classic pastime of assembling parts and working towards inclusion. Finally, we realized that gamification proved to be an effective strategy for teachers to make learning more engaging and accessible, enabling students to explore more complex concepts in the field of quantum mechanics, which are elucidated in the theoretical construction of this work.CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível SuperiorCNPq - Conselho Nacional de Desenvolvimento Científico e TecnológicoFAPEMIG - Fundação de Amparo a Pesquisa do Estado de Minas GeraisTrabalho de Conclusão de Curso (Graduação)O ensino de física é, por si só, desafiador. Entretanto, com a crescente popularidade da mecânica quântica, a temática é por vezes associada a contextos não científicos, o que torna urgente a necessidade de difundir conhecimentos básicos sobre física de sistemas quânticos ainda na educação básica. Em razão disso, este estudo visa promover uma introdução acessível e descomplicada aos sistemas de spins interagentes, utilizando o modelo de Heisenberg para analisar cadeias de spins. Para isso, construímos uma fundamentação teórica que ampara a diagonalização de matrizes e o estudo de fenômenos quânticos, inclusive abordando o Efeito Kondo. Ademais, a partir da criação e uso de programas computacionais, obtivemos resultados numéricos exatos que nos permitiram interpretar quantidades físicas relevantes, que foram fundamentais para a contextualização dos sistemas interagentes. Tudo isso é contextualizado para os alunos da educação básica utilizando peças de encaixe palpáveis e coloridas para ilustrar conceitos do mundo quântico. Essa abordagem se enquadra à metodologia de gamificação, que torna o aprendizado mais envolvente e acessível aos alunos. Os discentes se tornam protagonistas da construção do saber aproveitando o passatempo clássico de montagem de peças e trabalhando a inclusão. Por fim, percebemos que a gamificação se mostrou uma estratégia eficaz para o docente tornar o aprendizado mais envolvente e acessível, habilitando os alunos a explorarem conceitos mais complexos no campo da mecânica quântica, que são elucidados na construção teórica desse trabalho