Numeric Reward Machines

Reward machines inform reinforcement learning agents about the reward structure of the environment and often drastically speed up the learning process. However, reward machines only accept Boolean features such as robot-reached-gold. Consequently, many inherently numeric tasks cannot profit from the guidance offered by reward machines. To address this gap, we aim to extend reward machines with numeric features such as distance-to-gold. For this, we present two types of reward machines: numeric-Boolean and numeric. In a numeric-Boolean reward machine, distance-to-gold is emulated by two Boolean features distance-to-gold-decreased and robot-reached-gold. In a numeric reward machine, distance-to-gold is used directly alongside the Boolean feature robot-reached-gold. We compare our new approaches to a baseline reward machine in the Craft domain, where the numeric feature is the agent-to-target distance. We use cross-product Q-learning, Q-learning with counter-factual experiences, and the options framework for learning. Our experimental results show that our new approaches significantly outperform the baseline approach. Extending reward machines with numeric features opens up new possibilities of using reward machines in inherently numeric tasks.Comment: ICAPS 2024; Workshop on Bridging the Gap Between AI Planning and Reinforcement Learnin

Liječenje subglotične stenoze: iskustvo jednog centra

The aim of this article is to present experiences of the Department of Otorhinolaryngology and Head and Neck Surgery, Zagreb University Hospital Center with the treatment of patients with subglottic stenosis. Subglottic stenosis is a rare congenital or acquired disorder of airway patency that is part of a wider complex of disorders known as laryngotracheal stenosis with the ultimate effect in the form of respiratory insufficiency that can be life-threatening. As an acquired condition, it is most often the result of iatrogenic damage to the larynx and trachea during invasive airway management, whether it is intubation or tracheotomy. In the case of intubation as the etiologic factor, cases of prolonged intubation were most common. Retrospective analysis of patient medical histories over a ten-year period was performed and 29 patients met the inclusion criteria. All patients were monitored for at least two years after completion of treatment. Out of a total of 29 treated patients, 20 were permanently decannulated, of which 4 have paresis of one or both vocal cords. In conclusion, there is no clear treatment protocol for patients with subglottic stenosis. The optimal modality of treatment is combined endoscopic and open surgical treatment.Cilj ovoga rada je prikazati iskustva Klinike za bolesti uha, nosa i grla i kirurgiju glave i vrata Kliničkog bolničkog centra Zagreb u liječenju bolesnika sa subglotičnom stenozom. Subglotična stenoza je rijedak prirođen ili stečeni poremećaj prohodnosti dišnog puta koji je dio šireg kompleksa poremećaja poznatog kao laringotrahealna stenoza s krajnjm učinkom u vidu respiracijske insuficijencije koja može biti opasna za život. Kao stečeno stanje najčešće je posljedica jatrogenog oštećenja grkljana i traheje prilikom invazivnog osiguranja dišnog puta, bilo da se radi o intubaciji ili traheotomiji. U slučaju intubacije kao etiološkog čimbenika najčešće se radi o slučajevima produljene intubacije. Učinjena je retrospektivna analiza povijesti bolesti bolesnika u desetogodišnjem razdoblju te je 29 bolesnika ispunjavalo kriterije uključenja u studiju. Svi bolesnici praćeni su najmanje dvije godine nakon završenog liječenja. Svi bolesnici liječeni su kirurški, endoskospki, otvorenom metodom ili kombinirano. Od ukupno 29 liječenih bolesnika 20 ih je trajno dekanilirano, od kojih 4 ima parezu jedne ili obje glasiljke. Zaključno, ne postoji jasan protokol liječenja bolesnika sa subglotičnom stenozom. Optimalan modalitet liječenja je kombinirano endoskopsko lasersko i otvoreno kirurško liječenje

A Survey on the Integration of Generative AI for Critical Thinking in Mobile Networks

In the near future, mobile networks are expected to broaden their services and coverage to accommodate a larger user base and diverse user needs. Thus, they will increasingly rely on artificial intelligence (AI) to manage network operation and control costs, undertaking complex decision-making roles. This shift will necessitate the application of techniques that incorporate critical thinking abilities, including reasoning and planning. Symbolic AI techniques already facilitate critical thinking based on existing knowledge. Yet, their use in telecommunications is hindered by the high cost of mostly manual curation of this knowledge and high computational complexity of reasoning tasks. At the same time, there is a spurt of innovations in industries such as telecommunications due to Generative AI (GenAI) technologies, operating independently of human-curated knowledge. However, their capacity for critical thinking remains uncertain. This paper aims to address this gap by examining the current status of GenAI algorithms with critical thinking capabilities and investigating their potential applications in telecom networks. Specifically, the aim of this study is to offer an introduction to the potential utilization of GenAI for critical thinking techniques in mobile networks, while also establishing a foundation for future research.Comment: 14 pages, 3 figures, 4 table

Intent-driven strategic tactical planning for autonomous site inspection using cooperative drones

Realization of industry-scale, goal-driven, autonomous systems with AI planning technology faces several challenges: flexibly specifying planning goal states in varying situations, synthesizing plans in large state spaces, re-planning in dynamic situations, and facilitating humans to supervise, give feedback and intervene. In this paper, we present Intent-driven Strategic Tactical Planning (ISTP) to address these challenges. We demonstrate its efficacy through its application for radio base station inspection across several locations using drones. The inspection task involves capturing images, thermal images or signal measurements - called knowledge-objects - of various components of the base stations for downstream processing. In the ISTP approach, an operator indicates her goals by flying the drone to different components of interest. These goals are generalized to capture the intent of the operator, which are then instantiated in new situations to generate goals dynamically. Towards planning and re-planning in large state spaces to achieve these goals efficiently, we extend the Strategic-Tactical Planning paradigm. All the components of ISTP are integrated in an intuitive UI and demonstrated through a real life use-case built on the UNITY simulator platform

Različiti pristupi marketinškoj komunikaciji - od klasičnog outbound do digitalnog inbound marketinga

Kao što je komunikacija temeljni element interakcije i stvaranja međuljudskih odnosa, tako je marketinška komunikacija ključna za izgradnju odnosa tvrtke s javnostima i njezin opstanak na tržištu. Tijekom stoljeća marketinška se komunikacija razvijala i rasla uzastopno s tehnologijama pa tvrtke danas imaju mogućnost interakcije sa svojim javnostima putem raznih alata u bilo kojem trenutku i kutku svijeta. Ovaj diplomski rad prati razvoj marketinške komunikacije od njezinih početaka do digitalnog doba. Radom su tako obuhvaćeni teorijski okviri marketinške komunikacije (oblici i elementi), zatim razvoj koji je doveo do nastanka integrirane marketinške komunikacije, te konačno do tehnologija koje su omogućile tvrtkama interakciju s korisnicima na digitalnoj platformi. Naravno, razvoj tehnologije je osim na alate marketinške komunikacije značajno utjecao i na pristup komunikaciji, stoga se danas prepoznaju dva osnovna pristupa; tradicionalni outbound i suvremeni inbound pristup. S ciljem analize u kojoj su se mjeri tvrtke prilagodile suvremenoj komunikaciji i kako je to utjecalo na njihov odnos s tržištem napravljeno je istraživanje na primjeru triju domaćih tvrtki iz sektora prehrambene industrije.As communication is the fundamental element of interaction and establishing interpersonal relationships, marketing communication is crucial for building a company's relationship with the public and its survival on the market. Throughout centuries, marketing communications have evolved and grown alongside with technology, and today companies have the opportunity to interact with their public through various tools at any point and in corner of the world. The paper follows the development of marketing communications from its beginning to the digital age. The paper covers the theoretical frameworks of marketing communication (forms and elements), the development that led to the emergence of integrated marketing communications, and the technologies that enabled companies to interact with users on a digital platform. Clearly, the development of technology, apart from the marketing communications tools, also had a significant impact on approach to communication, so today there are two basic approaches; a traditional outbound and modern inbound approach. With the aim of analysing the extent of adapting companies to modern communication and how it affected the company-public relationship, a research that included three FMCG companies was conducted

Prevention of money laundering through banks

U ovome radu obraĊena je tema sprjeĉavanja pranja novca putem banaka. Opisane su faze i metode pranja novca, zakonska prava i obveze banaka, sudionici, naĉini i postupci sprjeĉavanja pranja novca i njihov utjecaj na cjelokupan financijski sustav, ne samo u Republici Hrvatskoj nego i meĊunarodno. Time se naglašava vaţnost meĊunarodne suradnje jer je pranje novca meĊunarodni problem koji uništava cjelokupni financijski sustav zemlje. Da bi bankarski sektor odrţao financijski sustav ĉistim, vaţno je da se pridrţava propisanih zakonskih odredbi, metoda i mjera koje mora provoditi. Samo pridrţavanjem navedenih radnji, bankarski sektor moţe izvršiti zadaću koja mu se stavlja na veliku odgovornost, jer sve daljnje kaznene aktivnosti zapoĉinju upravo na pranju novca. Nedvojbeno, svi sudionici u sustavu sprjeĉavanja pranja novca i financiranja terorizma moraju meĊusobno suraĊivati i razmjenjivati informacije kako bi bili pravodobno informirani te si meĊusobno pomogli u iskorjenjivanju pranja novca.This paper deals with the topic of the prevention of money laundering and terrorist financing in the banking system. The paper describes the stages and methods of money laundering, legal rights and obligations of banks, participants, methods and procedures of money laundering prevention and their impact on the entire financial system, not only in the Republic of Croatia but also internationally. This emphasizes the importance of international cooperation since money laundering is an international problem. In order to keep the financial sector clean, it is relevant that it adheres to the statutory provisions, methods and measures it must implement. It is only by adhering to these actions that the banking sector can carry out the task that is being held to great responsibility, since all further criminal activities begin precisely on money laundering. Undoubtedly, all participants in the system for the prevention of money laundering and terrorist financing must cooperate with each other, exchange information in order to be informed in a timely manner and help each other to eradicate money laundering

Task Allocation Optimization for Multicore Embedded Systems

Modern embedded systems are becoming increasingly performance intensive, since, on the one hand, they include more complex functionality than before, and one the other hand, the functionality that was typically realized with hardware is often moved to software. Multicore technology, previously successfully used for general-purpose systems, is penetrating into the domain of embedded systems. While it does increase the performance capacity, it also introduces the problem of how to allocate software tasks to the cores of the hardware platform, as different allocations exhibit different extra-functional properties. An intuitive example is allocating too many tasks to a core --- the core will be overloaded and tasks will miss their deadlines. This thesis addresses the issue of task allocation in multicore embedded systems. The overall goal of the thesis is to advance the way soft real-time multicore systems are developed, by providing new methods and tools that enable deciding already at design-time which task to run on which core, with respect to a number of timing-related extra-functional properties. To achieve this goal, we developed a model-based framework for task allocation optimization. The framework uses model simulation in order to obtain performance predictions for particular task allocations. This in turn enables testing a large number of allocation candidates in search for one that exhibits good timing-related performance. Apart from defining and implementing the framework, three additional contributions are provided, each tackling a particular aspect of the framework: the influence of task allocation on communication duration is studied and interpreted in the context of design-time model-based analysis; a novel heuristic for guiding task allocation optimization is defined; and finally, a novel optimization method combining performance prediction and performance measurement is defined