This article presents a new Fast Hash-based File Existence Checking (FHFEC) method for archiving ... more This article presents a new Fast Hash-based File Existence Checking (FHFEC) method for archiving systems. During the archiving process, there are many submissions which are actually unchanged files that do not need to be re-archived. In this system, instead of comparing the entire files, only digests of the files are compared. Strong cryptographic hash functions with a low probability of collision can be used as digests. We propose a fast algorithm to check if a certain hash, that is, a corresponding file, is already stored in the system. The algorithm is based on dividing the whole domain of hashes into equally sized regions, and on the existence of a pointer array, which has exactly one pointer for each region. Each pointer points to the location of the first stored hash from the corresponding region and has a null value if no hash from that region exists. The entire structure can be stored in random access memory or, alternatively, on a dedicated hard disk. A statistical performance analysis has been performed that shows that in certain cases FHFEC performs nearly optimally. Extensive simulations have confirmed these analytical results. The performance of FHFEC has been compared to the performance of a binary search (BIS) and B+tree, which are commonly used in file systems and databases for table indices. The results show that FHFEC significantly outperforms both of them.
The amount of data created annually is growing exponentially. Some of the newly created data is i... more The amount of data created annually is growing exponentially. Some of the newly created data is important enough to be kept for shorter or longer time, therefore the need for data archiving is growing proportionally to data growth. We designed an efficient and scalable archiving system that can easily be adapted to the needs of different archiving applications and scenarios. A Fast Hash-based File Existence Checking algorithm is the core of the system. It out-performs other algorithms for file uniqueness checking. The proposed archiving system is a distributed processing and storage system that works with metadata hashes and file hashes. We implemented the archiving system in a form of a backup-like application for files on personal computers in the laboratory. The application operational tests yielded very favourable results, for example, the need for archive storage almost halved just because of the elimination of files with exactly the same content. We expect the proposed archiving system will find many uses in the present and future data archiving efforts.
When developing new network protocols, new functionalities of network devices, new traffic models... more When developing new network protocols, new functionalities of network devices, new traffic models and other novelties, we face the problem of non-existing tools. If we want to simulate the behavior of the network with a newly developed feature we have to develop a new simulator or add a new functionality to an existing simulation tool. Both options can be quite complex and/or time consuming. To facilitate the testing and simulation of a new packet scheduler that we developed during our research we have developed a simulator for a general model of a network device. For this purpose we have used the Modular Simulation language, MODSIM III. It is a general-purpose, modular, block structured language that provides support for object-oriented programming, discrete event simulation and animated graphics. The simulator includes modules for the most important elements and functions of a packet network device, modules for collection of results, and modules for writing the results into a standard format files. We have tested and validated the operation of the simulator with analytically verifiable settings. The simulation and analytical results were practically the same. Encouraged with that we have then simulated newly developed packet schedulers and network device functionalities. Some of the simulation results are presented in this article, more you can find in corresponding references.
Delay is an important Quality of Service (QoS) parameter. A significant part of an end-to-end del... more Delay is an important Quality of Service (QoS) parameter. A significant part of an end-to-end delay of a data flow is latency – the delay induced by scheduling algorithms in network devices. In this paper we present a latency analysis of a Deficit Round-Robin (DRR) scheduler. We correct inaccuracies and deficiencies of the DRR latency analysis detected in works of other authors. We derive a new and more precise latency bound of the DRR scheduler and show that, contrary to the latency bounds derived by other authors, our bound is mathematically correct. Finally we give a comparative analysis of the DRR latency bounds discussed in the paper. Natančnejša meja zakasnitve odpravnika z deficitarnim krožnim dodeljevanjem Povzetek-Zakasnitev je pomemben parameter kakovosti storitve. Pomemben del zakasnitev od-konca-do-konca za posamezen podatkovni pretok je tudi začetna zakasnitev ali latenca – zakasnitev al-goritma odpravnika v omrežnih napravah. V tem prispevku predstavljamo analizo latence odpravnika z deficitarnim krožnim dodeljevanjem (Deficit Round-Robin-DRR). Osvetlimo in odpravimo nepravil-nosti in pomanjkljivosti analiz latence odpravnika DRR v delih drugih avtorjev. Izpeljemo novo in natančnejšo mejo zakasnitve odpravnika DRR in pokažemo, da je le-ta, v nasprotju z mejami zakas-nitve drugih avtorjev, matematično pravilna. Za konec podamo primerjalno analizo mej zakasnitev odpravnikov DRR, obravnavanih v tem prispevku. Ključne besede: DRR, odpravnik z deficitarnim krožnim dodeljevanjem, latenca, meja zakasnitve, paketno omrežje, odpravnik
— Sorting is extensively used in many applications. Predominantly, comparison based sequential so... more — Sorting is extensively used in many applications. Predominantly, comparison based sequential sorting algorithms are used. High speed computing is also driving the quest for ever faster sorting. Sorting networks executing parallel sorting and dataflow computational paradigm are offered as a possible solution. Sorting networks are briefly explained, Details are given for the Bitonic merge sort algorithm, which we used in our experiments. Sorting was implemented on an entry model of the Maxeler dataflow supercomputing systems. We employed different testing scenarios. Results show, that for a small size array of 128 numbers, the speedup, comparing to the fastest sorting algorithm on a CPU, achieves factor of 16. Moving to more advanced Maxeler systems, we expect to be able to sort larger arrays and achieve much greater speedups.
The paper discusses the shift in the computing paradigm and the programming model for Big Data pr... more The paper discusses the shift in the computing paradigm and the programming model for Big Data problems and applications. We compare DataFlow and ControlFlow programming models through their quantity and quality aspects. Big Data problems and applications that are suitable for implementation on DataFlow computers should not be measured using the same measures as ControlFlow computers. We propose a new methodology for benchmarking, which takes into account not only the execution time, but also the power and space, needed to complete the task. Recent research shows that if the TOP500 ranking was based on the new performance measures, DataFlow machines would outperform ControlFlow machines. To support the above claims, we present eight recent implementations of various algorithms using the DataFlow paradigm, which show considerable speed-ups, power reductions and space savings over their implementation using the ControlFlow paradigm.
This article studies the suitability of smartphones with built-in inertial sensors for biofeedbac... more This article studies the suitability of smartphones with built-in inertial sensors for biofeedback applications. Biofeedback systems use various sensors to measure body functions and parameters. These sensor data are analyzed, and the results are communicated back to the user, who then tries to act on the feedback signals. Smartphone inertial sensors can be used to capture body movements in biomechanical biofeedback systems. These sensors exhibit various inaccuracies that induce significant angular and positional errors. We studied deterministic and random errors of smartphone accelerometers and gyroscopes, primarily focusing on their biases. Based on extensive measurements, we determined accelerometer and gyroscope noise models and bias variation ranges. Then, we compiled a table of predicted positional and angular errors under various biofeedback system operation conditions. We suggest several bias compensation options that are suitable for various examples of use in real-time biofeedback applications. Measurements within the developed experimental biofeedback application show that under certain conditions, even uncompensated sensors can be used for real-time biofeedback. For general use, especially for more demanding biofeedback applications, sensor biases should be compensated. We are convinced that real-time biofeedback systems based on smartphone inertial sensors are applicable to many similar examples in sports, healthcare, and other areas.
TENCON, IEEE Region 10 International Conference, 2001
The Internet traditionally provides service that is commonly characterized as a best-effort servi... more The Internet traditionally provides service that is commonly characterized as a best-effort service. Many applications run very well using this service model but some new interactive applications such as telephony or video conferencing impose stringent demands to the network. We discuss a proposal for changing the architecture of IP networks in order to provide some level of service discrimination. Architecture
This paper presents a new Fast Hash-based File Existence Checking (FHFEC) method for archiving sy... more This paper presents a new Fast Hash-based File Existence Checking (FHFEC) method for archiving systems. During the archiving process, there are many submissions which are actually unchanged files that do not need to be re-archived. In this system, instead of comparing the entire files, only digests of the files are compared. Strong cryptographic hash functions with a low probability of collision can be used as digests. We propose a fast algorithm to check if a certain hash, i.e. a corresponding file, is already stored in the system. The algorithm is based on dividing the whole domain of hashes into equally sized regions, and on the existence of a pointer array, which has exactly one pointer for each region. Each pointer points to the location of the first stored hash from the corresponding region and has a null value if no hash from that region exists. The entire structure can be stored in random access memory or, alternatively, on a dedicated hard disk. A statistical performance analysis has been performed which shows that in certain cases FHFEC performs nearly optimally. Extensive simulations have confirmed these analytical results. The performance of FHFEC has been compared to the performance of a binary search (BIS) and B+tree, which are commonly used in file systems and databases for table indices. The results show that FHFEC significantly outperforms both of them.
International Journal of Distributed Sensor Networks, 2015
Big data processing is becoming a reality in numerous real-world applications. With the emergence... more Big data processing is becoming a reality in numerous real-world applications. With the emergence of new data intensive technologies and increasing amounts of data, new computing concepts are needed. The integration of big data producing technologies, such as wireless sensor networks, Internet of Things, and cloud computing, into cyber-physical systems is reducing the available time to find the appropriate solutions. This paper presents one possible solution for the coming exascale big data processing: a data flow computing concept. The performance of data flow systems that are processing big data should not be measured with the measures defined for the prevailing control flow systems. A new benchmarking methodology is proposed, which integrates the performance issues of speed, area, and power needed to execute the task. The computer ranking would look different if the new benchmarking methodologies were used; data flow systems would outperform control flow systems. This statement is backed by the recent results gained from implementations of specialized algorithms and applications in data flow systems. They show considerable factors of speedup, space savings, and power reductions regarding the implementations of the same in control flow computers. In our view, the next step of data flow computing development should be a move from specialized to more general algorithms and applications.
VOIP (Voice over IP) uses unreliable IP networks, which basically do not offer any performance gu... more VOIP (Voice over IP) uses unreliable IP networks, which basically do not offer any performance guarantees for the transmission of real-time voice frames. In this paper we study the transmission of VOIP over Ethernet in two steps. First we try to evaluate the general performance of Ethernet using theoretical analysis and then we compare it to simulation results. We have chosen most widely adopted standards and technologies used for that purpose: Ethernet 10Base-T LAN and G.723.1 speech coder. As delay is the key performance parameter for real-time speech transmission, the most attention is given to its study at different network loads. Gained results are quite encouraging, showing a possibility of a satisfactory speech transmission up to a certain network load that is usually not reached at normal Ethernet operation conditions.
New services for the "real-time Internet" are introduced and compared. Special concern is given t... more New services for the "real-time Internet" are introduced and compared. Special concern is given to the last mile of an end-to-end transmission path -dial-up and LAN connections. Extensive simulations of voice transmission over the Ethernet have been carried out with the most interesting results presented and commented.
Possibilities of incorporating existing LAN technologies into communication networks of the futur... more Possibilities of incorporating existing LAN technologies into communication networks of the future are analysed in this article. Basic properties of future communication networks and types of traffic they will have to carry are explained in short. The most widespread LAN technologies, their division, basic properties and capabilities to form an integral part of the future communication networks are also discussed. Special attention is given to Ethernet technology and the transmission of real-time voice over it. Its suitability for the transmission of G.729 coded speech is determined by simulation and guidelines for using Ethernet as a network for real-time communications are given.
Povzetek. Včlanku so podane definicije kakovosti storitve, ki se nanašajo na komunikacijska omrež... more Povzetek. Včlanku so podane definicije kakovosti storitve, ki se nanašajo na komunikacijska omrežja in storitve. Osvetliti poskušamo kakovost storitve s stališč uporabnika, ki se opira na subjektivne mere in kakovost izkušnje ter ponudnika, za katerega so pomembne objektivne mere kakovosti storitve. V nadaljevanju se osredotočimo na objektivne mere določanja kakovosti in množico med seboj odvisnih parametrov kakovosti storitve. Podane so tudi nekatere standardizirane in splošne vrednosti parametrov kakovosti storitve za zahtevne aplikacije. Za razumevanje uvajanja kakovosti storitve v paketna omrežja so na kratko podane osnovne lastnosti paketnih omrežij in pomembnost vzdrževanja dogovorjene stopnje kakovosti storitve na celotni prenosni poti. Ne nazadnje pa v javnih omrežjih ne smemo zanemariti niti psihološkega in sociološkega vidika uvajanja različnih stopenj kakovosti storitve.
This article presents a new Fast Hash-based File Existence Checking (FHFEC) method for archiving ... more This article presents a new Fast Hash-based File Existence Checking (FHFEC) method for archiving systems. During the archiving process, there are many submissions which are actually unchanged files that do not need to be re-archived. In this system, instead of comparing the entire files, only digests of the files are compared. Strong cryptographic hash functions with a low probability of collision can be used as digests. We propose a fast algorithm to check if a certain hash, that is, a corresponding file, is already stored in the system. The algorithm is based on dividing the whole domain of hashes into equally sized regions, and on the existence of a pointer array, which has exactly one pointer for each region. Each pointer points to the location of the first stored hash from the corresponding region and has a null value if no hash from that region exists. The entire structure can be stored in random access memory or, alternatively, on a dedicated hard disk. A statistical performance analysis has been performed that shows that in certain cases FHFEC performs nearly optimally. Extensive simulations have confirmed these analytical results. The performance of FHFEC has been compared to the performance of a binary search (BIS) and B+tree, which are commonly used in file systems and databases for table indices. The results show that FHFEC significantly outperforms both of them.
The amount of data created annually is growing exponentially. Some of the newly created data is i... more The amount of data created annually is growing exponentially. Some of the newly created data is important enough to be kept for shorter or longer time, therefore the need for data archiving is growing proportionally to data growth. We designed an efficient and scalable archiving system that can easily be adapted to the needs of different archiving applications and scenarios. A Fast Hash-based File Existence Checking algorithm is the core of the system. It out-performs other algorithms for file uniqueness checking. The proposed archiving system is a distributed processing and storage system that works with metadata hashes and file hashes. We implemented the archiving system in a form of a backup-like application for files on personal computers in the laboratory. The application operational tests yielded very favourable results, for example, the need for archive storage almost halved just because of the elimination of files with exactly the same content. We expect the proposed archiving system will find many uses in the present and future data archiving efforts.
When developing new network protocols, new functionalities of network devices, new traffic models... more When developing new network protocols, new functionalities of network devices, new traffic models and other novelties, we face the problem of non-existing tools. If we want to simulate the behavior of the network with a newly developed feature we have to develop a new simulator or add a new functionality to an existing simulation tool. Both options can be quite complex and/or time consuming. To facilitate the testing and simulation of a new packet scheduler that we developed during our research we have developed a simulator for a general model of a network device. For this purpose we have used the Modular Simulation language, MODSIM III. It is a general-purpose, modular, block structured language that provides support for object-oriented programming, discrete event simulation and animated graphics. The simulator includes modules for the most important elements and functions of a packet network device, modules for collection of results, and modules for writing the results into a standard format files. We have tested and validated the operation of the simulator with analytically verifiable settings. The simulation and analytical results were practically the same. Encouraged with that we have then simulated newly developed packet schedulers and network device functionalities. Some of the simulation results are presented in this article, more you can find in corresponding references.
Delay is an important Quality of Service (QoS) parameter. A significant part of an end-to-end del... more Delay is an important Quality of Service (QoS) parameter. A significant part of an end-to-end delay of a data flow is latency – the delay induced by scheduling algorithms in network devices. In this paper we present a latency analysis of a Deficit Round-Robin (DRR) scheduler. We correct inaccuracies and deficiencies of the DRR latency analysis detected in works of other authors. We derive a new and more precise latency bound of the DRR scheduler and show that, contrary to the latency bounds derived by other authors, our bound is mathematically correct. Finally we give a comparative analysis of the DRR latency bounds discussed in the paper. Natančnejša meja zakasnitve odpravnika z deficitarnim krožnim dodeljevanjem Povzetek-Zakasnitev je pomemben parameter kakovosti storitve. Pomemben del zakasnitev od-konca-do-konca za posamezen podatkovni pretok je tudi začetna zakasnitev ali latenca – zakasnitev al-goritma odpravnika v omrežnih napravah. V tem prispevku predstavljamo analizo latence odpravnika z deficitarnim krožnim dodeljevanjem (Deficit Round-Robin-DRR). Osvetlimo in odpravimo nepravil-nosti in pomanjkljivosti analiz latence odpravnika DRR v delih drugih avtorjev. Izpeljemo novo in natančnejšo mejo zakasnitve odpravnika DRR in pokažemo, da je le-ta, v nasprotju z mejami zakas-nitve drugih avtorjev, matematično pravilna. Za konec podamo primerjalno analizo mej zakasnitev odpravnikov DRR, obravnavanih v tem prispevku. Ključne besede: DRR, odpravnik z deficitarnim krožnim dodeljevanjem, latenca, meja zakasnitve, paketno omrežje, odpravnik
— Sorting is extensively used in many applications. Predominantly, comparison based sequential so... more — Sorting is extensively used in many applications. Predominantly, comparison based sequential sorting algorithms are used. High speed computing is also driving the quest for ever faster sorting. Sorting networks executing parallel sorting and dataflow computational paradigm are offered as a possible solution. Sorting networks are briefly explained, Details are given for the Bitonic merge sort algorithm, which we used in our experiments. Sorting was implemented on an entry model of the Maxeler dataflow supercomputing systems. We employed different testing scenarios. Results show, that for a small size array of 128 numbers, the speedup, comparing to the fastest sorting algorithm on a CPU, achieves factor of 16. Moving to more advanced Maxeler systems, we expect to be able to sort larger arrays and achieve much greater speedups.
The paper discusses the shift in the computing paradigm and the programming model for Big Data pr... more The paper discusses the shift in the computing paradigm and the programming model for Big Data problems and applications. We compare DataFlow and ControlFlow programming models through their quantity and quality aspects. Big Data problems and applications that are suitable for implementation on DataFlow computers should not be measured using the same measures as ControlFlow computers. We propose a new methodology for benchmarking, which takes into account not only the execution time, but also the power and space, needed to complete the task. Recent research shows that if the TOP500 ranking was based on the new performance measures, DataFlow machines would outperform ControlFlow machines. To support the above claims, we present eight recent implementations of various algorithms using the DataFlow paradigm, which show considerable speed-ups, power reductions and space savings over their implementation using the ControlFlow paradigm.
This article studies the suitability of smartphones with built-in inertial sensors for biofeedbac... more This article studies the suitability of smartphones with built-in inertial sensors for biofeedback applications. Biofeedback systems use various sensors to measure body functions and parameters. These sensor data are analyzed, and the results are communicated back to the user, who then tries to act on the feedback signals. Smartphone inertial sensors can be used to capture body movements in biomechanical biofeedback systems. These sensors exhibit various inaccuracies that induce significant angular and positional errors. We studied deterministic and random errors of smartphone accelerometers and gyroscopes, primarily focusing on their biases. Based on extensive measurements, we determined accelerometer and gyroscope noise models and bias variation ranges. Then, we compiled a table of predicted positional and angular errors under various biofeedback system operation conditions. We suggest several bias compensation options that are suitable for various examples of use in real-time biofeedback applications. Measurements within the developed experimental biofeedback application show that under certain conditions, even uncompensated sensors can be used for real-time biofeedback. For general use, especially for more demanding biofeedback applications, sensor biases should be compensated. We are convinced that real-time biofeedback systems based on smartphone inertial sensors are applicable to many similar examples in sports, healthcare, and other areas.
TENCON, IEEE Region 10 International Conference, 2001
The Internet traditionally provides service that is commonly characterized as a best-effort servi... more The Internet traditionally provides service that is commonly characterized as a best-effort service. Many applications run very well using this service model but some new interactive applications such as telephony or video conferencing impose stringent demands to the network. We discuss a proposal for changing the architecture of IP networks in order to provide some level of service discrimination. Architecture
This paper presents a new Fast Hash-based File Existence Checking (FHFEC) method for archiving sy... more This paper presents a new Fast Hash-based File Existence Checking (FHFEC) method for archiving systems. During the archiving process, there are many submissions which are actually unchanged files that do not need to be re-archived. In this system, instead of comparing the entire files, only digests of the files are compared. Strong cryptographic hash functions with a low probability of collision can be used as digests. We propose a fast algorithm to check if a certain hash, i.e. a corresponding file, is already stored in the system. The algorithm is based on dividing the whole domain of hashes into equally sized regions, and on the existence of a pointer array, which has exactly one pointer for each region. Each pointer points to the location of the first stored hash from the corresponding region and has a null value if no hash from that region exists. The entire structure can be stored in random access memory or, alternatively, on a dedicated hard disk. A statistical performance analysis has been performed which shows that in certain cases FHFEC performs nearly optimally. Extensive simulations have confirmed these analytical results. The performance of FHFEC has been compared to the performance of a binary search (BIS) and B+tree, which are commonly used in file systems and databases for table indices. The results show that FHFEC significantly outperforms both of them.
International Journal of Distributed Sensor Networks, 2015
Big data processing is becoming a reality in numerous real-world applications. With the emergence... more Big data processing is becoming a reality in numerous real-world applications. With the emergence of new data intensive technologies and increasing amounts of data, new computing concepts are needed. The integration of big data producing technologies, such as wireless sensor networks, Internet of Things, and cloud computing, into cyber-physical systems is reducing the available time to find the appropriate solutions. This paper presents one possible solution for the coming exascale big data processing: a data flow computing concept. The performance of data flow systems that are processing big data should not be measured with the measures defined for the prevailing control flow systems. A new benchmarking methodology is proposed, which integrates the performance issues of speed, area, and power needed to execute the task. The computer ranking would look different if the new benchmarking methodologies were used; data flow systems would outperform control flow systems. This statement is backed by the recent results gained from implementations of specialized algorithms and applications in data flow systems. They show considerable factors of speedup, space savings, and power reductions regarding the implementations of the same in control flow computers. In our view, the next step of data flow computing development should be a move from specialized to more general algorithms and applications.
VOIP (Voice over IP) uses unreliable IP networks, which basically do not offer any performance gu... more VOIP (Voice over IP) uses unreliable IP networks, which basically do not offer any performance guarantees for the transmission of real-time voice frames. In this paper we study the transmission of VOIP over Ethernet in two steps. First we try to evaluate the general performance of Ethernet using theoretical analysis and then we compare it to simulation results. We have chosen most widely adopted standards and technologies used for that purpose: Ethernet 10Base-T LAN and G.723.1 speech coder. As delay is the key performance parameter for real-time speech transmission, the most attention is given to its study at different network loads. Gained results are quite encouraging, showing a possibility of a satisfactory speech transmission up to a certain network load that is usually not reached at normal Ethernet operation conditions.
New services for the "real-time Internet" are introduced and compared. Special concern is given t... more New services for the "real-time Internet" are introduced and compared. Special concern is given to the last mile of an end-to-end transmission path -dial-up and LAN connections. Extensive simulations of voice transmission over the Ethernet have been carried out with the most interesting results presented and commented.
Possibilities of incorporating existing LAN technologies into communication networks of the futur... more Possibilities of incorporating existing LAN technologies into communication networks of the future are analysed in this article. Basic properties of future communication networks and types of traffic they will have to carry are explained in short. The most widespread LAN technologies, their division, basic properties and capabilities to form an integral part of the future communication networks are also discussed. Special attention is given to Ethernet technology and the transmission of real-time voice over it. Its suitability for the transmission of G.729 coded speech is determined by simulation and guidelines for using Ethernet as a network for real-time communications are given.
Povzetek. Včlanku so podane definicije kakovosti storitve, ki se nanašajo na komunikacijska omrež... more Povzetek. Včlanku so podane definicije kakovosti storitve, ki se nanašajo na komunikacijska omrežja in storitve. Osvetliti poskušamo kakovost storitve s stališč uporabnika, ki se opira na subjektivne mere in kakovost izkušnje ter ponudnika, za katerega so pomembne objektivne mere kakovosti storitve. V nadaljevanju se osredotočimo na objektivne mere določanja kakovosti in množico med seboj odvisnih parametrov kakovosti storitve. Podane so tudi nekatere standardizirane in splošne vrednosti parametrov kakovosti storitve za zahtevne aplikacije. Za razumevanje uvajanja kakovosti storitve v paketna omrežja so na kratko podane osnovne lastnosti paketnih omrežij in pomembnost vzdrževanja dogovorjene stopnje kakovosti storitve na celotni prenosni poti. Ne nazadnje pa v javnih omrežjih ne smemo zanemariti niti psihološkega in sociološkega vidika uvajanja različnih stopenj kakovosti storitve.
Uploads
Papers by Anton Kos