Learning to Switch Between Machines and Humans

Reinforcement learning agents have been mostly developed and evaluated under the assumption that they will operate in a fully autonomous manner -- they will take all actions. In this work, our goal is to develop algorithms that, by learning to switch control between machine and human agents, allow existing reinforcement learning agents to operate under different automation levels. To this end, we first formally define the problem of learning to switch control among agents in a team via a 2-layer Markov decision process. Then, we develop an online learning algorithm that uses upper confidence bounds on the agents' policies and the environment's transition probabilities to find a sequence of switching policies. We prove that the total regret of our algorithm with respect to the optimal switching policy is sublinear in the number of learning steps. Moreover, we also show that our algorithm can be used to find multiple sequences of switching policies across several independent teams of agents operating in similar environments, where it greatly benefits from maintaining shared confidence bounds for the environments' transition probabilities. Simulation experiments in obstacle avoidance in a semi-autonomous driving scenario illustrate our theoretical findings and demonstrate that, by exploiting the specific structure of the problem, our proposed algorithm is superior to problem-agnostic algorithms.Comment: Added support for unknown transition probabilities and multiple team

A Critical Analysis of the Form and Concept of Poems of Secondary High Schools Persian Literarute Books

In the curriculum of Iran, the Persian Literature lessons and its textbooks play a significant role. At the same time, poetry is very important in Persian literature. For these reasons, this paper intends to describe analytically the Persian Literature books of this period in a quantitative and qualitative way to examine the status of poetry and the intellectual and aesthetic approaches of the selected poems. We want to know if the poems in these books have the ability to portray an appropriate, transcendent, and realistic picture of Persian poetry in students’ minds. The results indicate that the books in question are mostly prose-based. The selected poems are mostly devoted to epic, patriotic and revolutionary literature, and from an intellectual and aesthetic point of view, they may not be attractive enough to encourage students to read and enjoy textbooks in class. Due to the inappropriate distribution of poems in literary genres, styles of poetry, poetry formats, and poems of Persian books, this course does not show the correct face of Persian poetry to the audience. In terms of the status of the poets and their works, there are also many criticisms of the three books. Therefore, this study emphasizes the necessity of revising and modifying the textbooks studied

Using the Prey-Predator Equation for the Water Allocation Problem and Its Comparison with Conventional Water Allocation Methods, A Case Study of The Atrak River Basin

Allocating the water resources in a basin to several stakeholders is a common issue at both national and international levels. Despite the many extensive studies carried out on the water allocation problem, a method still needs to be developed for the equitable and sustainable allocation of water to all the stakeholders in a shared basin. Over the last few decades, a number of mathematical methods such as the Nash bargaining, area monotonic, equal loss, and Kalai-Smorodinsky solutions have been applied to the problem of conflict resolution that are collectively known as optimization methods, each one yielding a single solution. In this study, a novel mathematical model based on the prey-predator equation is employed for water allocation to resolve conflicts among stakeholders in the agricultural sector. The advantage of the proposed model lies in its capability to calculate balanced allocation of irrigation water to stakeholders aimed at the sustainable development of the region. The model calculates the stakeholders’ profits and payoffs and determines their interactions in a time series. Finally, the model is employed for resolving conflicts in the Atrak River basin in the northeast of Iran which is now facing a serious water tension. Comparison of the results obtained from the proposed model and those from four conventional conflict resolution methods applied to the same basin implies the superiority of the proposed model in yielding dynamic solutions rather single ones

Forecasting and Analysis of Monthly Rainfalls in Ardabil Province by Arima, Autoregrressive, and Winters Models

Introduction: Rainfall has the highest variability at time and place scale. Rainfall fluctuation in different geographical areas reveals the necessity of investigating this climate element and suitable models to forecast the rate of precipitation for regional planning. Ardabil province has always faced rainfall fluctuations and shortage of water supply. Precipitation is one of the most important features of the environment. The amount of precipitation over time and in different places is subject to large fluctuations which may be periodical. Studies show that, due to the certain complexities of rainfall, the models which used to predict future values will also need greater accuracy and less error. Among the forecasting models, Arima has more applications and it has replaced with other models. Materials and Methods: In this research, through order 2 Autoregrressive, Winters, and Arima models, monthly rainfalls of Ardabil synoptic station (representing Ardabil province) for a 31-year period (1977-2007) were investigated. To assess the presence or absence of significant changes in mean precipitation of Ardabil synoptic station, rainfall of this station was divided into two periods: 1977-1993 and 1994-2010. T-test was used to statistically examine the difference between the two periods. After adjusting the data, descriptive statistics were applied. In order to model the total monthly precipitation of Ardabil synoptic station, Winters, Autoregressive, and Arima models were used. Among different models, the best options were chosen to predict the time series including the mean absolute deviation (MAD), the mean squared errors (MSE), root mean square errors (RMSE) and mean absolute percentage errors (MAPE). In order to select the best model among the available options under investigation, the predicted value of the deviation of the actual value was utilized for the months of 2006-2010. Results and Discussion: Statistical characteristics of the total monthly precipitation in Ardabil synoptic station indicates that in May, the highest and in August, the lowest monthly total rainfall accounted in this station. Standard deviation of rainfall reached to the lowest level in August and its peak in November. Coefficients of skewness and kurtosis of total rainfall in all seasons, indicates a lack of compliance with normal distribution. From the view of the range of total monthly rainfall, October and August have highest and the lowest tolerance in these parameters, respectively. The results showed that the percentage of the mean absolute error for Arima, Winters and Autoregressive models was 61.82, 148.39 and 81.54 respectively and its R square came to be 88.28, 61.07 and 85.12 respectively. The comparison of the parameters is an indication of the fact that Arima has the highest R square and the lowest mean absolute error of 88.28 and 61.82 respectively than Winters and Autoregressive models. The presence or absence of significant changes in mean precipitation during 1977-1993 and 2010-1994 in Ardabil synoptic station shows that the difference of rainfall is not significant at the 5% error level from statistical point of view. The comparison between the monthly mean rainfall of Ardabil synoptic station in 1994-2010 and 1977-1993 indicates that rainfall has somewhat decreased in the former in recent years. Considering the low average monthly rainfall of Ardabil synoptic station in 1994-2010 compared to 1977-1993 (21.98 versus 26.11 mm), although no statistically significant difference was found in the average rainfall, low rainfall in this station would not be unexpected in the coming years. The comparison of predicted and actual values from 2011 to 2013 in Ardabil synoptic station showed that fitting real data with expected data was relatively acceptable. The observed differences between the actual and predicted values can be related to the influence of rainfalls and many local and dynamical factors of this area. Therefore, it is necessary for climatologists to better explain and predict phenomena besides statistical models and pay more attention to general circulation models (GCM) under different climate conditions. Conclusion: Results of rainfall investigation by order 2 Autoregrressive, Winters, and Arima models showed a descending trend in monthly rainfalls in the coming years across the study location. The results of modeling and analysis of monthly rainfalls in Ardabil synoptic station showed that among these models, Arima was better than the other two because it enjoyed the lowest MAPE and the highest R2. AIC, RMSE and MAD scales of different patterns were calculated and finally, SARIMA(1,1,1)(2,0,1)12 pattern having the lowest AIC, RMSE and MAD was selected as the most appropriate pattern for monthly rainfall forecasting in Ardabil synoptic station