Design for real-time data acquisition based on streaming technology

For the LHD project a long-pulse plasma experiment of 1-h duration is planned. In this quasi steady-state operation, the data acquisition system will be required to continuously transfer the diagnostic data from the digitizer front-end and display them in real-time. The CompactPCI standard is used to replace the conventional CAMAC digitizers in LHD, because it provides good functionality for real-time data streaming and also a connectivity with modern PC technology. The digitizer scheme, interface to the host computer, adoption of data compression, and downstream applications are discussed in detail to design and implement this new real-time data streaming system for LHD plasma diagnostics

Multi-Layer Distributed Storage of LHD Plasma Diagnostic Database

At the end of LHD experimental campaign in 2003, the amount of whole plasma diagnostics raw data had reached 3.16 GB in a long-pulse experiment. This is a new world record in fusion plasma experiments, far beyond the previous value of 1.5 GB/shot. The total size of the LHD diagnostic data is about 21.6 TB for the whole six years of experiments, and it continues to grow at an increasing rate. The LHD diagnostic database and storage system, i.e. the LABCOM system, has a completely distributed architecture to be sufficiently flexible and easily expandable to maintain integrity of the total amount of data. It has three categories of the storage layer: OODBMS volumes in data acquisition servers, RAID servers, and mass storage systems, such as MO jukeboxes and DVD-R changers. These are equally accessible through the network. By data migration between them, they can be considered a virtual OODB extension area. Their data contents have been listed in a “facilitator” PostgreSQL RDBMS, which now contains about 6.2 million entries, and informs the optimized priority to clients requesting data. Using the “glib” compression for all of the binary data and applying the three-tier application model for the OODB data transfer/retrieval, an optimized OODB read-out rate of 1.7 MB/s and effective client access speed of 3?25 MB/s have been achieved. As a result, the LABCOM data system has succeeded in combination of the use of RDBMS, OODBMS, RAID, and MSS to enable a virtual and always expandable storage volume, simultaneously with rapid data access

Steady-state data acquisition method for LHD diagnostics

The LHD experiment has gone through 5 campaign periods over the past 4 years, during which the diagnostics data continues to grow and the primary 28 measurements produce about 620 MB/shot in 150 shot/day 3-min cycles. In 2002, 30-min long-pulse experiments will be carried out in LHD, where real-time operations are indispensable for plasma measurements and data acquisition. The new scheme for utilizing conventional CAMAC digitizers in long-pulse experiments has been discussed and examined. As a result, in LHD, CAMACs will shift into 120?180 s cyclic operation, synchronized by the diagnostic timing system. The new CompactPCI-based digitizer frontend has performed about 84 MB/s continuous acquisition in benchmarks, and has been formulated with the conventional CAMAC system to make concurrent acquisitions

Nonstop Lose-Less Data Acquisition and Storing Method for Plasma Motion Images

Plasma diagnostic data analysis often requires the original raw data as they are, in other words, at the same frame rate and resolution of the CCD camera sensor. As a non-interlace VGA camera typically generates over 70 MB/s video stream, usual frame grabber cards apply the lossy compression encoder, such as mpeg-1/-2 or mpeg-4, to drastically lessen the bit rate. In this study, a new approach, which makes it possible to acquire and store such the wideband video stream without any quality reduction, has been successfully achieved. Simultaneously, the real-time video streaming is even possible at the original frame rate. For minimising the exclusive access time in every data storing, it has adopted the directory structure to hold every frame files separately, instead of one long consecutive file. The popular ‘zip’ archive method improves the portability of data files, however, the JPEG-LS image compression is applied inside by replacing its intrinsic deflate/inflate algorithm that has less performances for image data

Adaptive data migration scheme with facilitator database and multi-tier distributed storage in LHD

Recent “data explosion” induces the demand for high flexibility of storage extension and data migration. The data amount of LHD plasma diagnostics has grown 4.6 times bigger than that of three years before. Frequent migration or replication between plenty of distributed storage becomes mandatory, and thus increases the human operational costs. To reduce them computationally, a new adaptive migration scheme has been developed on LHD’s multi-tier distributed storage. So-called the HSM (Hierarchical Storage Management) software usually adopts a low-level cache mechanism or simple watermarks for triggering the data stage-in and out between two storage devices. However, the new scheme can deal with a number of distributed storage by the facilitator database that manages the whole data locations with their access histories and retrieval priorities. Not only the inter-tier migration but also the intra-tier replication and moving are even manageable so that it can be a big help in extending or replacing storage equipment. The access history of each data object is also utilized to optimize the volume size of fast and costly RAID, in addition to a normal cache effect for frequently retrieved data. The new scheme has been verified its effectiveness so that LHD multi-tier distributed storage and other next-generation experiments can obtain such the flexible expandability

コンテキストを説明する メタデータ ～核融合実験データの実例から～

ORCID　0000-0001-6388-4489開催概要 名称 JAPAN OPEN SCIENCE SUMMIT 2023 会期 2023年6月19日（月）〜 6月23日（金） 会場 WEBオンライン 主催 国立情報学研究所、科学技術振興機構、物質・材料研究機構、科学技術・学術政策研究所、情報通信研究機構、学術資源リポジトリ協議会、情報知識学会、合同会社AMANEconference objec

"Plasma and Fusion Cloud" Data analysis environment

ORCID　0000-0002-0441-6340The Fusion Cloud concept is currently underway at the National Institute for Fusion Science (NIFS). This concept is to deploy data collection and data analysis systems, which are assets developed through LHD experiments, to other laboratories in Japan through a high-speed network. The Open Data Server discussed in this paper is part of the Fusion Cloud concept, and data collected at LHD is currently being made publicly available. In the future, the data collected by various devices will be made widely available for use in fusion research, and will also be used for research beyond fusion and across different fields. In addition, in the future, not only the data but also a portable environment for analyzing these data will be provided on the cloud.journal articl

High Energy Particle Measurements during Long Discharge in LHD

The spatial resolved energy spectra can be observed during a long discharge of NBI plasma bycontinuously scanning the neutral particle analyzer. In these discharges, the plasmas are initiated by the ECH heating, after that NBI#2 (Co-injection) sustains the plasma during 40-60 seconds. The scanned pitch angle is from 44 degrees to 74 degrees. The injected neutral beam (hydrogen) energy of NBI#2 is only 130 keV because the original ion source polarity is negative. The shape of spectra is almost similar from 44 degrees to 53 degrees. However the spectra from 55 degrees are strongly varied. It reflects the injection pitch angle of the beam according to the simulation (53 degrees ot R* = 3.75 m in simulation). The beam keeps the pitch angle at incidence until the beam energy becomes to the energy, which the pitch angle scattering is occurred by the energy loss due to the electron collision. The low flux region can be observed around 10-15 keV, which is 15 times of the electron temperature. The energy region may be equal to the energy at which the pitch angle scattering is occurred. At the energy, the particle is scattered by the collision with the plasma ions and some of particles may run away from the plasma because they have a possibility to enter the loss cone. According to the simulation, the loss cone can be expected at the 10 keV with the small angular dependence. The depth of the loss cone is deep at the small pitch angle. The hollow in the spectrum may be concluded to be the loss cone as the tendency is almost agreed with the experimental result