Identifying the Machine Learning Family from Black-Box Models

[EN] We address the novel question of determining which kind of machine learning model is behind the predictions when we interact with a black-box model. This may allow us to identify families of techniques whose models exhibit similar vulnerabilities and strengths. In our method, we first consider how an adversary can systematically query a given black-box model (oracle) to label an artificially-generated dataset. This labelled dataset is then used for training different surrogate models (each one trying to imitate the oracle¿s behaviour). The method has two different approaches. First, we assume that the family of the surrogate model that achieves the maximum Kappa metric against the oracle labels corresponds to the family of the oracle model. The other approach, based on machine learning, consists in learning a meta-model that is able to predict the model family of a new black-box model. We compare these two approaches experimentally, giving us insight about how explanatory and predictable our concept of family is.This material is based upon work supported by the Air Force Office of Scientific Research under award number FA9550-17-1-0287, the EU (FEDER), and the Spanish MINECO under grant TIN 2015-69175-C4-1-R, the Generalitat Valenciana PROMETEOII/2015/013. F. Martinez-Plumed was also supported by INCIBE under grant INCIBEI-2015-27345 (Ayudas para la excelencia de los equipos de investigacion avanzada en ciberseguridad). J. H-Orallo also received a Salvador de Madariaga grant (PRX17/00467) from the Spanish MECD for a research stay at the CFI, Cambridge, and a BEST grant (BEST/2017/045) from the GVA for another research stay at the CFI.Fabra-Boluda, R.; Ferri Ramírez, C.; Hernández-Orallo, J.; Martínez-Plumed, F.; Ramírez Quintana, MJ. (2018). Identifying the Machine Learning Family from Black-Box Models. Lecture Notes in Computer Science. 11160:55-65. https://doi.org/10.1007/978-3-030-00374-6_6S556511160Angluin, D.: Queries and concept learning. Mach. Learn. 2(4), 319–342 (1988)Benedek, G.M., Itai, A.: Learnability with respect to fixed distributions. Theor. Comput. Sci. 86(2), 377–389 (1991)Biggio, B., et al.: Security Evaluation of support vector machines in adversarial environments. In: Ma, Y., Guo, G. (eds.) Support Vector Machines Applications, pp. 105–153. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-02300-7_4Blanco-Vega, R., Hernández-Orallo, J., Ramírez-Quintana, M.J.: Analysing the trade-off between comprehensibility and accuracy in mimetic models. In: Suzuki, E., Arikawa, S. (eds.) DS 2004. LNCS (LNAI), vol. 3245, pp. 338–346. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30214-8_29Dalvi, N., Domingos, P., Sanghai, S., Verma, D., et al.: Adversarial classification. In: Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 99–108. ACM (2004)Dheeru, D., Karra Taniskidou, E.: UCI machine learning repository (2017). http://archive.ics.uci.edu/mlDomingos, P.: Knowledge discovery via multiple models. Intell. Data Anal. 2(3), 187–202 (1998)Duin, R.P.W., Loog, M., Pȩkalska, E., Tax, D.M.J.: Feature-based dissimilarity space classification. In: Ünay, D., Çataltepe, Z., Aksoy, S. (eds.) ICPR 2010. LNCS, vol. 6388, pp. 46–55. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-17711-8_5Fernández-Delgado, M., Cernadas, E., Barro, S., Amorim, D.: Do we need hundreds of classifiers to solve real world classification problems. J. Mach. Learn. Res. 15(1), 3133–3181 (2014)Ferri, C., Hernández-Orallo, J., Modroiu, R.: An experimental comparison of performance measures for classification. Pattern Recognit. Lett. 30(1), 27–38 (2009)Giacinto, G., Perdisci, R., Del Rio, M., Roli, F.: Intrusion detection in computer networks by a modular ensemble of one-class classifiers. Inf. Fusion 9(1), 69–82 (2008)Huang, L., Joseph, A.D., Nelson, B., Rubinstein, B.I., Tygar, J.: Adversarial machine learning. In: Proceedings of the 4th ACM Workshop on Security and Artificial Intelligence, pp. 43–58 (2011)Kuncheva, L.I., Whitaker, C.J.: Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach. Learn. 51(2), 181–207 (2003)Landis, J.R., Koch, G.G.: An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics 33, 363–374 (1977)Lowd, D., Meek, C.: Adversarial learning. In: Proceedings of the 11th ACM SIGKDD International Conference on Knowledge Discovery in Data mining, pp. 641–647. ACM (2005)Martınez-Plumed, F., Prudêncio, R.B., Martınez-Usó, A., Hernández-Orallo, J.: Making sense of item response theory in machine learning. In: Proceedings of 22nd European Conference on Artificial Intelligence (ECAI). Frontiers in Artificial Intelligence and Applications, vol. 285, pp. 1140–1148 (2016)Papernot, N., McDaniel, P., Goodfellow, I.: Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277 (2016)Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z.B., Swami, A.: The limitations of deep learning in adversarial settings. In: 2016 IEEE European Symposium on Security and Privacy (EuroS&P), pp. 372–387. IEEE (2016)Papernot, N., McDaniel, P., Wu, X., Jha, S., Swami, A.: Distillation as a defense to adversarial perturbations against deep neural networks. In: 2016 IEEE Symposium on Security and Privacy (SP), pp. 582–597. IEEE (2016)Sesmero, M.P., Ledezma, A.I., Sanchis, A.: Generating ensembles of heterogeneous classifiers using stacked generalization. Wiley Interdiscip. Rev.: Data Min. Knowl. Discov. 5(1), 21–34 (2015)Smith, M.R., Martinez, T., Giraud-Carrier, C.: An instance level analysis of data complexity. Mach. Learn. 95(2), 225–256 (2014)Tramèr, F., Zhang, F., Juels, A., Reiter, M.K., Ristenpart, T.: Stealing machine learning models via prediction APIs. In: USENIX Security Symposium, pp. 601–618 (2016)Valiant, L.G.: A theory of the learnable. Commun. ACM 27(11), 1134–1142 (1984)Wallace, C.S., Boulton, D.M.: An information measure for classification. Comput. J. 11(2), 185–194 (1968)Wolpert, D.H.: Stacked generalization. Neural Netw. 5(2), 241–259 (1992

Wdr18 Is Required for Kupffer's Vesicle Formation and Regulation of Body Asymmetry in Zebrafish

Correct specification of the left-right (L-R) axis is important for organ morphogenesis. Conserved mechanisms involving cilia rotation inside node-like structures and asymmetric Nodal signaling in the lateral plate mesoderm (LPM), which are important symmetry-breaking events, have been intensively studied. In zebrafish, the clustering and migration of dorsal forerunner cells (DFCs) is critical for the formation of the Kuppfer's vesicle (KV). However, molecular events underlying DFC clustering and migration are less understood. The WD-repeat proteins function in a variety of biological processes, including cytoskeleton assembly, intracellular trafficking, mRNA splicing, transcriptional regulation and cell migration. However, little is known about the function of WD-repeat proteins in L-R asymmetry determination. Here, we report the identification and functional analyses of zebrafish wdr18, a novel gene that encodes a WD-repeat protein that is highly conserved among vertebrate species. wdr18 was identified from a Tol2 transposon-mediated enhancer trap screen. Follow-up analysis of wdr18 mRNA expression showed that it was detected in DFCs or the KV progenitor cells and later in the KV at early somitogenesis stages. Morpholino knockdown of wdr18 resulted in laterality defects in the visceral organs, which were preceded by the mis-expression of Nodal-related genes, including spaw and pitx2. Examination of morphants at earlier stages revealed that the KV had fewer and shorter cilia which are immotile and a smaller cavity. We further investigated the organization of DFCs in wdr18 morphant embryos using ntl and sox17 as specific markers and found that the clustering and migration of DFC was altered, leading to a disorganized KV. Finally, through a combination of wdr18 and itgb1b morpholino injections, we provided evidence that wdr18 and itgb1b genetically interact in the laterality determination process. Thus, we reveal a new and essential role for WD-repeat proteins in the determination and regulation of L-R asymmetry and propose a potential mechanism for wdr18 in the regulation of DFC clustering and migration and KV formation

Age of Child, More than HPV Type, Is Associated with Clinical Course in Recurrent Respiratory Papillomatosis

Background: RRP is a devastating disease in which papillomas in the airway cause hoarseness and breathing difficulty. The disease is caused by human papillomavirus (HPV), 6 or 11 and is very variable. Patients undergo multiple surgeries to maintain a patent airway and in order to communicate vocally. Several small studies have been published in which most have noted that HPV 11 is associated with a more aggressive course. Methodology/Principal Findings: Papilloma biopsies were taken from patients undergoing surgical treatment of RRP and were subjected to HPV typing. 118 patients with juvenile-onset RRP with a least 1 year of clinical data and infected with a single HPV type were analyzed. HPV 11 was encountered in 40% of the patients. By our definition, most of the patients in the sample (81%) had run an aggressive course. The odds of a patient with HPV 11 running an aggressive course were 3.9 times higher that that of patients with HPV 6 (Fisher's exact p=0.017). However, clinical course was more closely associated with age of the patient (at diagnosis and at the time of the current surgery) than with HPV type. Patients with HPV 11 were diagnosed at a younger age (2.4y) than were those with HPV 6 (3.4y) (p=0.014). Both by multiple linear regression and by multiple logistics regression HPV type was only weakly associated with metrics of disease course when simultaneously accounting for age. Conclusions/Significance Abstract: The course of RRP is variable and a quarter of the variability can be accounted for by the age of the patient. HPV 11 is more closely associated with a younger age at diagnosis than it is associated with an aggressive clinical course. These data suggest that there are factors other than HPV type and age of the patient that determine disease course. © 2008 Buchinsky et al

Long-Distance Signals Are Required for Morphogenesis of the Regenerating Xenopus Tadpole Tail, as Shown by Femtosecond-Laser Ablation

tadpoles has recently emerged as an important model for these studies; we explored the role of the spinal cord during tadpole tail regeneration.Using ultrafast lasers to ablate cells, and Geometric Morphometrics to quantitatively analyze regenerate morphology, we explored the influence of different cell populations. For at least twenty-four hours after amputation (hpa), laser-induced damage to the dorsal midline affected the morphology of the regenerated tail; damage induced 48 hpa or later did not. Targeting different positions along the anterior-posterior (AP) axis caused different shape changes in the regenerate. Interestingly, damaging two positions affected regenerate morphology in a qualitatively different way than did damaging either position alone. Quantitative comparison of regenerate shapes provided strong evidence against a gradient and for the existence of position-specific morphogenetic information along the entire AP axis.We infer that there is a conduit of morphology-influencing information that requires a continuous dorsal midline, particularly an undamaged spinal cord. Contrary to expectation, this information is not in a gradient and it is not localized to the regeneration bud. We present a model of morphogenetic information flow from tissue undamaged by amputation and conclude that studies of information coming from far outside the amputation plane and regeneration bud will be critical for understanding regeneration and for translating fundamental understanding into biomedical approaches

pitx2 Deficiency Results in Abnormal Ocular and Craniofacial Development in Zebrafish

Human PITX2 mutations are associated with Axenfeld-Rieger syndrome, an autosomal-dominant developmental disorder that involves ocular anterior segment defects, dental hypoplasia, craniofacial dysmorphism and umbilical abnormalities. Characterization of the PITX2 pathway and identification of the mechanisms underlying the anomalies associated with PITX2 deficiency is important for better understanding of normal development and disease; studies of pitx2 function in animal models can facilitate these analyses. A knockdown of pitx2 in zebrafish was generated using a morpholino that targeted all known alternative transcripts of the pitx2 gene; morphant embryos generated with the pitx2ex4/5 splicing-blocking oligomer produced abnormal transcripts predicted to encode truncated pitx2 proteins lacking the third (recognition) helix of the DNA-binding homeodomain. The morphological phenotype of pitx2ex4/5 morphants included small head and eyes, jaw abnormalities and pericardial edema; lethality was observed at ∼6–8-dpf. Cartilage staining revealed a reduction in size and an abnormal shape/position of the elements of the mandibular and hyoid pharyngeal arches; the ceratobranchial arches were also decreased in size. Histological and marker analyses of the misshapen eyes of the pitx2ex4/5 morphants identified anterior segment dysgenesis and disordered hyaloid vasculature. In summary, we demonstrate that pitx2 is essential for proper eye and craniofacial development in zebrafish and, therefore, that PITX2/pitx2 function is conserved in vertebrates

Recreational and occupational field exposure to freshwater cyanobacteria – a review of anecdotal and case reports, epidemiological studies and the challenges for epidemiologic assessment

Cyanobacteria are common inhabitants of freshwater lakes and reservoirs throughout the world. Under favourable conditions, certain cyanobacteria can dominate the phytoplankton within a waterbody and form nuisance blooms. Case reports and anecdotal references dating from 1949 describe a range of illnesses associated with recreational exposure to cyanobacteria: hay fever-like symptoms, pruritic skin rashes and gastro-intestinal symptoms are most frequently reported. Some papers give convincing descriptions of allergic reactions while others describe more serious acute illnesses, with symptoms such as severe headache, pneumonia, fever, myalgia, vertigo and blistering in the mouth. A coroner in the United States found that a teenage boy died as a result of accidentally ingesting a neurotoxic cyanotoxin from a golf course pond. This death is the first recorded human fatality attributed to recreational exposure to cyanobacteria, although uncertainties surround the forensic identification of the suspected cyanotoxin in this case. We systematically reviewed the literature on recreational exposure to freshwater cyanobacteria. Epidemiological data are limited, with six studies conducted since 1990. Statistically significant increases in symptoms were reported in individuals exposed to cyanobacteria compared to unexposed counterparts in two Australian cohort studies, though minor morbidity appeared to be the main finding. The four other small studies (three from the UK, one Australian) did not report any significant association. However, the potential for serious injury or death remains, as freshwater cyanobacteria under bloom conditions are capable of producing potent toxins that cause specific and severe dysfunction to hepatic or central nervous systems. The exposure route for these toxins is oral, from ingestion of recreational water, and possibly by inhalation. A range of freshwater microbial agents may cause acute conditions that present with features that resemble illnesses attributed to contact with cyanobacteria and, conversely, acute illness resulting from exposure to cyanobacteria or cyanotoxins in recreational waters could be misdiagnosed. Accurately assessing exposure to cyanobacteria in recreational waters is difficult and unreliable at present, as specific biomarkers are unavailable. However, diagnosis of cyanobacteria-related illness should be considered for individuals presenting with acute illness following freshwater contact if a description is given of a waterbody visibly affected by planktonic mass development

Special cases : moons, rings, comets, trojans

Non-planetary bodies provide valuable insight into our current under- standing of planetary formation and evolution. Although these objects are challeng- ing to detect and characterize, the potential information to be drawn from them has motivated various searches through a number of techniques. Here, we briefly review the current status in the search of moons, rings, comets, and trojans in exoplanet systems and suggest what future discoveries may occur in the near future.Comment: Invited review (status August 2017