图 书 馆
并行RDF流处理查询的性能优化Optimizing the Performance for Concurrent RDF Stream Processing Queries |
|
| 课程网址: | http://videolectures.net/eswc2017_le_van_stream_processing_querie... |
| 主讲教师: | Chan Le Van |
| 开课单位: | 数据分析洞察中心 |
| 开课时间: | 2017-07-10 |
| 课程语种: | 英语 |
| 中文简介: |  随着物联网（IoT）和传感技术的日益普及，正在以非常快的速度生成大量数据流。为了探索物联网和语义技术集成的潜力，提供了一些RDF流处理（RSP）查询引擎，它们能够实时处理，分析和推理语义数据流。这样，RSP减轻了数据互操作性问题，并促进了对时间敏感的应用程序的知识发现和智能决策。但是，广泛使用RSP系统的主要障碍是查询性能。特别地，RSP引擎处理大量并发查询的能力非常有限，这限制了大规模流处理应用程序（例如，智能城市应用程序）来采用RSP。在本文中，我们提出了一种基于共享联接的方法，以提高用于并行查询的RSP引擎的性能。我们还利用查询联合机制来允许在多个RSP引擎实例上进行分布式查询，以提高并发查询和分布式查询的性能。我们应用负载平衡策略来分发查询，并进一步优化并发查询性能。我们通过扩展CQELS RSP引擎来提供概念验证的实现，并使用RSP的现有基准数据集评估我们的方法。我们还将我们提出的方法的性能与CQELS RSP引擎的最新实现方式进行了比较。 |
| 课程简介: | With the growing popularity of Internet of Things (IoT) and sensing technologies, a large number of data streams are being generated at a very rapid pace. To explore the potentials of the integration of IoT and semantic technologies, a few RDF Stream Processing (RSP) query engines are made available which are capable of processing, analyzing and reasoning over semantic data streams in real-time. This way, RSP mitigates data interoperability issues and promotes knowledge discovery and smart decision making for time-sensitive applications. However, a major hurdle in the wide adoption of RSP systems is their query performance. Particularly, the ability of RSP engines to handle a large number of concurrent queries is very limited which refrains large scale stream processing applications (e.g. smart city applications) to adopt RSP. In this paper, we propose a shared-join based approach to improve the performance of an RSP engine for concurrent queries. We also leverage query federation mechanisms to allow distributed query processing over multiple RSP engine instances in order to gain performance for concurrent and distributed queries. We apply load balancing strategies to distribute queries and further optimize the concurrent query performance. We provide a proof of concept implementation by extending CQELS RSP engine and evaluate our approach using existing benchmark datasets for RSP. We also compare the performance of our proposed approach with the state of the art implementation of CQELS RSP engine. |
| 关 键 词: | 物联网; 性能查询; 数据集 |
| 课程来源: | 视频讲座网 |
| 数据采集: | 2021-03-10：zyk |
| 最后编审: | 2021-03-10：zyk |
| 阅读次数: | 64 |