VEDLIOT – Accelerated AIoT. Jens Hagemeyer. 2nd Workshop on Deep Learning for IoT (DL4IoT), co-located with HiPEAC 2023, Toulouse, France, January 2023
Report
Share
Report
Share
1 of 30
More Related Content
Similar to HiPEAC2023-DL4IoT Workshop_Jean Hagemeyer presentation
The document outlines an agenda for a presentation on the VEDLIoT project. The agenda includes an introduction to VEDLIoT by Pedro Trancoso, a presentation on VEDLIoT Hardware Platforms by Kevin Mika, and a discussion of Performance Evaluation and Benchmarking in VEDLIoT by Mario Pormann. The VEDLIoT project aims to develop very efficient deep learning techniques for IoT applications through the use of heterogeneous hardware platforms and accelerators.
White Box Hardware Challenges in the 5G & IoT Hyperconnected EraCharo Sanchez
The development of an agile mobile network that supports a massive number of connected devices, low latencies, broadband speeds, network slicing, and edge intelligence is the result of a number of technologies that form the 5G vision. Advantech 5G Edge Servers and Universal Edge Appliances have been designed for the network edge to meet high availability network needs providing an open virtual infrastructure for seamless network transformation toward cloud native 5G architectures. From SD-WAN and private networks to virtual RAN, Central Office and Edge Cloud, Advantech is enabling the co-creation of products and services that will form the backbone of the new 5G & IoT economy.
www.advantech.com/nc/spotlight/5G
The Considerations for Internet of Things @ 2017Jian-Hong Pan
物聯網是一門透過通訊,將端點蒐集到的資料,集中關聯分析,並將分析結果用以決策並回饋的工程藝術。
本次的分享將從物聯網的目的當作進入點,接著分享可能的佈署架構。並概述目前各個常用的通訊標準、協定,以及其所屬的角色。
除此之外,也會分享去年到柏林參加Linux Foundation舉辦的Open IoT Summit Europe 2016的心得。
在此,帶回一些國外對於物聯網節點的佈署、更新或維護的看法、作法。
另外,也會分享一些物聯網可能需要考量的資訊安全議題。
IoT is a kind of engineering art, which analyzes the collected data from
the device nodes through the communication and has the result for the
decision making and feedback.
This sharing goes for the purpose of IoT and it's deployment structure.
Then, the slide introduces the most used communication standards or
protocols in IoT and their roles.
Besides, also shares what I have got from the Open IoT Summit Europe 2016
which was held by Linux Foundation in Berlin last year.
It introduces how will the device nodes be deployed, updated and maintained.
Finally, the slide provides some security issues that should be considered
in IoT.
IEEE CS Phoenix - Internet of Things Innovations & Megatrends UpdateMark Goldstein
Mark Goldstein, President of International Research Center explored the next Internet wave, the Internet of Things (IoT), expected to connect tens of billions of new sensors and devices in the coming years. Waves of change will roll through home, business, government, industrial, medical, transportation, and other complex ecosystems. Mark examined how IoT will be implemented and monetized creating new business models from pervasive sensor deployments and data gathering, accompanied by new privacy and security risks. Explore IoT’s roadblocks and operational challenges, emerging standards and protocols, gateway and wireless integration, and big data strategies and opportunities in this presentation.
An end-to-end standard oneM2M infrastructure for the Smart Home - Andre Bottaromfrancis
OSGi Community Event 2015
A new world of applications emerges in the home from the growing variety of things – devices, sensors, actuators – potentially available. Several application domains are considered, e.g., security, energy efficiency, comfort, ambient assisted living, multimedia communication. The Smart Home is slowly taking off.</p>
Several actors exploit a new technical and economic opportunity to catalyze this market. This opportunity is based on the re-use of the infrastructure that telecom operators have deployed for today classic Internet and TV services. It raises technical and business challenges: Telecom operators have to open their home infrastructure to third-party applications while guaranteeing application security and consistency to all home business actors using this infrastructure.
Telecom operators have to open APIs at least two levels of their architecture: APIs in the cloud and APIs on an embedded device environment. This end-to-end infrastructure between the home network and service platforms has also to provide security at several levels, especially a consistent access right management.
The presentation will provide a vision of an open end-to-end architecture providing APIs in the cloud and in a home box to host any application and connect to any device in the Home. Among the standard organizations and industrial alliances, oneM2M standard specifications are making a reference architecture emerge. The implementation of oneM2M standard features in OSGi technology will be detailed, especially the end-to-end access right management discriminating both applications and users when accessing devices.
This infrastructure is currently prototyped thanks to the integration of open source software bricks provided by <a>Open the Box</a>, <a>Eclipse SmartHome</a> and <a>Eclipse OM2M</a> open initiatives.
IEEE CS Phoenix - Internet of Things Innovations & Megatrends 12/2/15Mark Goldstein
Mark Goldstein, President, International Research Center (http://www.researchedge.com/) presented on the Internet of Things Innovations & Megatrends exploring the next Internet wave, the Internet of Things (IoT), expected to connect tens of billions of new sensors and devices in the coming years. Waves of change will roll through home, business, government, industrial, medical, transportation, and other complex ecosystems. This presentation examines how IoT will be implemented and monetized creating new business models from pervasive sensor deployments and data gathering, accompanied by new privacy and security risks. And it explores IoT’s roadblocks and operational challenges, emerging standards and protocols, gateway and wireless integration, and big data strategies and opportunities.
Walking through the fog (computing) - Keynote talk at Italian Networking Work...FBK CREATE-NET
"Walking through the fog (computing): trends, use-cases and open issues"
Despite its huge success in many IT-enabled application scenarios, cloud computing has demonstrated some intrinsic limitations that may severely limit its adoption in several contexts where constraints like e.g. preserving data locally, ensuring real-time reactivity or guaranteeing operation continuity despite lack of Internet connectivity (or a combination of them) are mandatory. These distinguishing requirements fostered an increased interest toward computing approaches that inherit the flexibility and adaptability of the cloud paradigm, while acting in proximity of a specific scenario. As a consequence, the emergence of this “proximity computing” approach has exploded into a plethora of architectural solutions (and novel terms) like fog computing, edge computing, dew computing, mist computing but also cloudlets, mobile cloud computing, mobile edge computing (and probably few others I may not be aware of…). The talk will initially make an attempt to introduce some clarity among these “foggy” definitions by proposing a taxonomy whose aim is to help identifying their peculiarities as well as their overlaps. Afterwards, the most important components of a generalized proximity computing architecture will be explained, followed by the description of few research works and use cases investigated within our Center and based on this emerging paradigm. An overview of open issues and interesting research directions will conclude the talk.
Mark Goldstein, President of International Research Center gave the opening keynote address “Internet of Things – Transformative Megatrends for Sustainability” to the IEEE Conference on Technologies for Sustainability (IEEE SusTech, http://sites.ieee.org/sustech/) on October 10, 2016 in Phoenix, AZ. He explored the next Internet wave, the Internet of Things (IoT), expected to connect tens of billions of new sensors and devices in the coming years driving sustainability while transforming home, business, government, industrial, medical, transportation, and other complex ecosystems. This deck examines how IoT will be implemented and monetized creating new business models from pervasive sensor deployments and data gathering, accompanied by new privacy and security risks. Explore IoT’s roadblocks and operational challenges, emerging standards and protocols, gateway and wireless integration, and big data strategies and opportunities.
IoT is a green field of new business opportunities. The ran has started…..
Everyware Device Cloud (EDC) is a full set of Operational Technologies available also as a service, which represent the fastest way to start an IoT business.
You can connect a Device to Cloud in 15 minutes.
With EDC A typical IoT project would take 2 to 6 months to go live and the ROI is really fast
.
5G-Slicer: An emulator for mobile IoT applications deployed over 5G network s...MoysisSymeonides
"5G-Slicer An emulator for mobile IoT applications deployed over 5G network slices" presentation at 2022 IEEE/ACM Seventh International Conference on Internet-of-Things Design and Implementation (IoTDI)
The document discusses several key challenges for the Internet of Things (IoT), including a lack of interoperability between current proprietary vertical solutions, issues around easy deployment and plug-and-play functionality at scale, and security concerns over privacy and data protection with networked devices. It also outlines some approaches to addressing these challenges through open standards, distributed intelligence, and turning sensor data into meaningful knowledge through tagging and semantics.
02/2017 Santa Clara, California: Networks of autonomous devices and their imp...Frank Alexander Reusch
Direct communication between IoT devices works without central control. The use of expensive gateways is therefore not a prerequisite for IoT. Gateways are inflexible, limited in scaling and an ideal target for hacker attacks. Lemonbeat has reached with this presentation that direct device communication is taken into account in future standard "Web of Things (WoT)". WoT also needs to consider future developments. For example, the self-learning mechanism in the edge area, shown in this lecture.
- Ankit Sarin has over 7 years of experience in embedded firmware development, hardware design, and integration testing. He has worked on projects in various domains including industrial automation, SCADA, rail, oil and gas, and consumer products.
- His skills include embedded C/C++, assembly language, various protocols and interfaces. He has experience with development tools and environments on 8/16/32 bit platforms.
- His most recent role is as a senior software engineer at Larsen & Toubro where he works on firmware development for solar inverters and railway products. Previously he has worked on projects for Invensys, Cognizant, and Philips.
Low Power Wide Area Networks (LPWA) are a new type of network designed for the Internet of Things (IoT). LPWA networks use low bandwidth and power to connect billions of devices globally at a low cost per device. They are well-suited for high device density applications where devices only need to send small amounts of data occasionally. By 2025, LPWA networks like Sigfox and LoRaWAN are projected to represent 20-25% of the global IoT connectivity market. Sigfox is the first global LPWA network operator, using an "Ultra Narrow Band" technology that allows for massive network capacity and high energy efficiency of connected devices.
Industrial IoT Mayhem? Java IoT Gateways to the RescueEurotech
Industrial IoT comes with great expectations for operational efficiency, promising improved asset utilization and productivity gains. IIoT challenges include reliability, security, low maintenance, long lifecycle, and integration into heterogeneous and fragmented systems. This session proposes some architectural patterns that can be leveraged to overcome these challenges. It introduces, at the center of the solution, Java-powered IoT gateways and modular IoT application frameworks such as the open source Eclipse Kura. Incorporating a live demonstration, the presentation highlights some of the latest Eclipse Kura features such as a pluggable device model for fieldbus protocols, visual data flow, and connectivity across various IoT cloud service providers.
JavaOne 2016 - Presentation by Dave Woodard and Walt Bowers
Xilinx provides adaptable acceleration platforms for data centers. Their Alveo product lineup includes the U280, U250, U200, and low-profile U50 accelerator cards. The cards feature FPGAs with up to 1.3 million logic cells and high-speed memory. Xilinx also offers the U25 SmartNIC which combines an FPGA, ARM CPU, and dual 25GbE ports. These platforms accelerate workloads such as AI, databases, storage, and networking using reconfigurable and adaptable hardware. Xilinx supports deployment from their devices to cloud platforms using a unified software stack.
This resume summarizes Pratik Panchal's experience as an embedded engineer. He has over 4 years of experience working with C and C++ on projects involving digital signage, energy meters, networking gateways, and vending machines. His roles have included application development, firmware programming, and functionality testing on platforms such as Linux, Windows, and microcontrollers from Texas Instruments and Microchip. He is seeking a position where he can apply his programming skills and experience developing embedded systems.
Creating a Step Change in Cyber Security | ISCF DSbD Business-led Demonstrato...KTN
John Goodacre, the Digital Security by Design (DSbD) Challenge Director at Innovate UK presents the background to the ISCF DSbD programme which aims to "Create a Step Change in Cyber Security".
Similar to HiPEAC2023-DL4IoT Workshop_Jean Hagemeyer presentation (20)
IoT Tech Expo 2023_Micha vor dem Berge presentationVEDLIoT Project
VEDLIoT Next Generation AIoT Applications. Micha vor dem Berge. VEDLIoT Conference Track co-located with IoT Tech Expo, Amsterdam, Netherlands, September 2023
Next generation accelerated AIoT systems and applications. Pedro Trancoso. Special Session on EU Projects, co-located with Computing Frontiers 2023, Bologna, Italy, May 2023
IoT Week 2022-NGIoT session_Micha vor dem Berge presentationVEDLIoT Project
This document discusses optimizing a smart home system using edge computing and machine learning. It describes using embedded accelerators like the Nvidia Jetson AGX and Xavier to distribute neural networks and machine learning models to devices around the home. These include a smart mirror, kitchen, door, and other devices. The goal is to optimize the models to increase energy efficiency and distribute the workloads across the edge devices. One focus is developing a smart mirror prototype that can recognize faces, objects and gestures using embedded accelerators like the t.RECS and u.RECS boards to analyze camera input and interact with users through voice and a virtual display.
Next Generation IoT Architectures_Hans SalomonssonVEDLIoT Project
VEDLIoT Toolchain for Efficient Deep Learning on heterogeneous hardware, Hans Salomonsson, EU-IoT Training Workshops Series – "Next Generation IoT Architectures”, November 2021
The document discusses hardware platforms and accelerators for VEDLIoT. It describes the VEDLIoT Hardware Platform as a heterogeneous, modular, and scalable microserver system that supports the IoT spectrum from embedded to edge to cloud. It then provides details on several platforms: the RECS|Box platform which uses Computer-on-Module standards to achieve flexibility and performance; the t.RECS platform optimized for local edge applications; and the uRECS embedded device platform that supports machine learning acceleration and communication interfaces. Diagrams and specifications are given for the architectures of these platforms.
VEDLIoT Cognitive IoT Hardware Platform. René Griessl. Workshop on Deep Learning for IoT (DL4IoT), co-located with HiPEAC 2022, Budapest, Hungary, June 2022
Security for VEDLIoT Components, from Cloud through Edge to IoT. Marcelo Pasin. Workshop on Deep Learning for IoT (DL4IoT), co-located with HiPEAC 2022, Budapest, Hungary, June 2022
Security and Robustness for VEDLIoT Components, from Cloud through Edge. Marcelo Pasin. VEDLIoT Conference Track co-located with IoT Tech Expo, Amsterdam, Netherlands, September 2023
Reconfigurable ML Accelerators in VEDLIoT. Marco Tassemeier. Workshop on Deep Learning for IoT (DL4IoT), co-located with HiPEAC 2022, Budapest, Hungary, June 2022
HiPEAC2022-DL4IoT workshop_ Muhammad Waqar AzharVEDLIoT Project
Co-design of DL Accelerators in VEDLIoT. Muhammad Waqar Azhar. Workshop on Deep Learning for IoT (DL4IoT), co-located with HiPEAC 2022, Budapest, Hungary, June 2022.
VEDLIoT at FPL'23_Accelerators for Heterogenous Computing in AIoTVEDLIoT Project
VEDLIoT took part in the 33rd International Conference on Field-Programmable Logic and Applications (FPL 2023), in Gothenburg, Sweden. René Griessl (UNIBI) presented VEDLIoT and our latest achievements in the Research Projects Event session, giving a presentation entitled "Accelerators for Heterogenous Computing in AIoT".
Types of Garden (Mughal and Buddhist style)saloniswain225
Garden is the place where, flower blooming on a plant ,aesthetic things are present like Topiary, Hedges, Arches and many more. Whereas, Botanical garden is an educational institution for scientific research as well as gathering information about different culture. Such as, Hindu, Mughal , Buddhist style.
Prototype Implementation of Non-Volatile Memory Support for RISC-V Keystone E...LenaYu2
Handling confidential information has become an increasingly important concern among many areas of society. However, current computing environments have been still vulnerable to various threats, and we should think they are untrusted.
Trusted Execution Environments (TEEs) have attracted attention because they can execute a program in a trusted environment constructed on an untrusted platform.
Particularly, the RISC-V Keystone is one of the interesting TEEs since it is a flexibly customizable and fully open-source platform. On the other hand, as same as other TEEs, it must also delegate I/O processing, such as file accesses, to a host OS, resulting in the expensive overhead. For this problem, we thought utilizing byte-addressable non-volatile memory (NVM) modules is a useful solution to handle persistent data objects for TEEs.
In this paper, we introduce a prototype implementation of NVM support for the Keystone. Additionally, we evaluate it on the Freedom U500 built on a VC707 FPGA dev kit.
https://ken.ieice.org/ken/paper/20210720TC4K/
Dalghren, Thorne and Stebbins System of Classification of AngiospermsGurjant Singh
The Dahlgren, Thorne, and Stebbins system of classification is a modern method for categorizing angiosperms (flowering plants) based on phylogenetic relationships. Developed by botanists Rolf Dahlgren, Robert Thorne, and G. Ledyard Stebbins, this system emphasizes evolutionary relationships and incorporates extensive morphological and molecular data. It aims to provide a more accurate reflection of the genetic and evolutionary connections among angiosperm families and orders, facilitating a better understanding of plant diversity and evolution. This classification system is a valuable tool for botanists, researchers, and horticulturists in studying and organizing the vast diversity of flowering plants.
Ethical considerations play a crucial role in research, ensuring the protection of participants and the integrity of the study. Here are some subject-specific ethical issues that researchers need
ScieNCE grade 08 Lesson 1 and 2 NLC.pptxJoanaBanasen1
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it................
just download it..............
Detecting and translating language ambiguity with multilingual LLMsBehrang Mehrparvar
Language is one of the most important landmarks in humans in history. However, most languages could be ambiguous, which means the same conveyed text or speech, results in different actions by different readers or listeners. In this project we propose a method to detect the ambiguity of a sentence using translation by multilingual LLMs. In this context, we hypothesize that a good machine translator should preserve the ambiguity of sentences in all target languages. Therefore, we investigate whether ambiguity is encoded in the hidden representation of a translation model or, instead, if only a single meaning is encoded. The potential applications of the proposed approach span i) detecting ambiguous sentences, ii) fine-tuning existing multilingual LLMs to preserve ambiguous information, and iii) developing AI systems that can generate ambiguity-free languages when needed.
Cause and solution of Water hyacinth (Terror of Bengal)saloniswain225
Water hyacinth is a buoyant plant that has a clump of leaves with squishy stalks arising from the lower feathery roots and it is an invasive species came from South America
Keys of Identification for Indian Wood: A Seminar ReportGurjant Singh
Identifying Indian wood involves recognizing key characteristics such as grain patterns, color, texture, hardness, and specific anatomical features. These identification keys include observing the wood's pores, growth rings, and resin canals, as well as its scent and weight. Understanding these features is essential for accurate wood identification, which is crucial for various applications in carpentry, furniture making, and conservation.
Additionally, the application of Convolutional Neural Networks (CNN) in wood identification has revolutionized this field. CNNs can analyze images of wood samples to identify species with high accuracy by learning and recognizing intricate patterns and features. This technological advancement not only enhances the precision of wood identification but also accelerates the process, making it more efficient for industry professionals and researchers alike.
TOPIC: INTRODUCTION TO FORENSIC SCIENCE.pptximansiipandeyy
This presentation, "Introduction to Forensic Science," offers a basic understanding of forensic science, including its history, why it's needed, and its main goals. It covers how forensic science helps solve crimes and its importance in the justice system. By the end, you'll have a clear idea of what forensic science is and why it's essential.
Principles of Colorimetry - Mastering the Art of Colour MeasurementColours Guide
Unlock the secrets of colour science with our comprehensive presentation on the principles of colorimetry. This guide delves into the critical aspects of colour measurement, essential for ensuring consistent quality and customer satisfaction across various industries.
Key Topics Covered:
- Introduction to Colorimetry
- Human Perception of Colour
- Importance of Metamerism and Colour Constancy
- Understanding Different Colour Spaces (RGB, CMYK, Lab)
Utilising Spectrophotometers and Colorimeters
- Implementing Quality Control for Colour Consistency
Learn how mastering these techniques can enhance product quality and meet customer expectations. Perfect for professionals in printing, textiles, paint, food, cosmetics, automotive, electronics, and more.
Read the full article: https://colours.guide/principles-of-colorimetry/
Visit www.colours.guide for more insights and resources on colour science.
2. 2
Platform
Hardware: Scalable, heterogeneous, distributed
Accelerators: Efficiency boost by FPGA and ASIC technology
Toolchain: Optimizing Deep Learning for IoT
Use cases
Industrial IoT
Automotive
Smart Home
Open call
10 projects covering a wide range of AIoT applications
Early use and evaluation of VEDLIoT technology
Very Efficient Deep Learning for IoT –
VEDLIoT
Call: H2020-ICT2020-1
Topic: ICT-56-2020 Next Generation Internet of Things
Duration: 1. November 2020 – 31. Oktober 2023
Coordinator: Bielefeld University (Germany)
Overall budget: 7 996 646.25 €
Consortium: 12 partners from 4 EU countries (Germany,
Poland, Portugal and Sweden) and one associated
country (Switzerland).
More info:
https://www.vedliot.eu/
https://twitter.com/VEDLIoT
https://www.linkedin.com/company/vedliot/
4. 4
VEDLIoT Hardware Platform
Heterogeneous, modular, scalable microserver system
Supporting the full spectrum of IoT from embedded over the edge towards the cloud
Different technology concepts for improving
x86
GPU
ML-ASIC
ARM v8
GPU
SoC
FPGA
SoC
RISC-V
FPGA
VEDLIOT Cognitive
IoT Platform
Performance
Cost-effectiveness
Maintainability
Reliability
Energy-Efficiency
Safety
5. 5
RECS Architecture – RECS|BOX
RECS Server Backplane (up to 15 Carriers)
Carrier (PCIe Expansion)
Carrier (High Performance)
e.g. GPU-Accelerator
Carrier (Low Power)
#3
#2
Microserver
(High Performance)
#1
Microserver
(Low Power)
#16
#3
#2
Microserver
(Low Power)
#1
High-Speed Low-Latency Network (PCIe, High-Speed Serial)
Compute Network (up to 40 GbE)
Management Network (KVM, Monitoring, …)
HDMI/USB
iPass+ HD
QSFP+
RJ45
Ext. Connectors
GPU
SoC
FPGA
SoC
ARM
Soc
Low-Power Microserver
(Apalis/Jetson)
x86 ARM v8
High-Performance Microserver (COM
Express)
FPGA SoC
High-Performance
Carrier
(up to 3 microservers)
Low-Power Carrier
(up to 16 microservers)
6. 6
t.RECS
t.RECS Edge Server
Optimized platform for
local / edge applications
Provide interfaces for
Video
Camera
Peripheral input (USB)
Combine FPGA and
GPU acceleration
Compact dimensions
1 RU, E-ATX form factor
(2 RU/ 3 RU for special cases)
RECS Architecture – t.RECS
Microserver #3
(COM-HPC Client)
Microserver #1
(COM-HPC Client)
Microserver #2
(COM-HPC Server)
Switched PCIe (Host to Host)
External
interfaces
PCIe
expansion
Ethernet (up to 10 GbE)
Management Network (KVM, Monitoring, …)
I/O (Camera, Display, Radar/Lidar, Audio)
9. 9
Peak performance values of specialized accelerators, provided by the vendors
(precisions varying from INT8 to FP32)
Peak Performance of DL Accelerators
Average efficiency at 1000 GOPS /W
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
[CELLRANGE]
1
10
100
1,000
10,000
100,000
1,000,000
10,000,000
0.01 0.1 1 10 100 1000
Performance
[GOPS]
Power [Watt]
ASIC
GPU
FPGA
Ultra Low Power
High Performance
Low Power
10. 10
Yolo v4 accelerator performance
Performance of Yolo v4 for different hardware platform has been evaluated
Performance measurement for other networks (Resnet, EfficientNet) available as well
11. 11
▪ VEDLIoT accelerators support a large variety
of reconfigurable architectures
▪ From small embedded FPGAs to large ACAPs
▪ Large design space for FPGA-based accelerators
▪ Dynamic hardware reconfiguration
▪ Adapt to changing requirements at run-time
▪ Change characteristics of DL-accelerator
▪ Trade-off between
power and performance, power and accuracy, etc.
▪ Inference and training on FPGA
▪ Supports quantization from int8 to float32
▪ DL and Deep Reinforcement Learning
Reconfigurable DL accelerators
12. 12
DL accelerator co-design
"FiBHA: Fixed Budget Hybrid CNN Accelerator", Fareed Qararyah, Muhammad Waqar Azhar, Pedro Trancoso, IEEE 34th International Symposium on Computer Architecture and High-
Performance Computing (SBAC-PAD 2022), Bordeaux, France, November 2–5 2022
Monolithic design
● One engine computes
all the core layers
● E.g. TPU
SEML
● One engine computes all
layers of the same type
● PW engine, DW engine
SESL
● One engine per layer
● E.g. FINN
FiBHA
● SESL + SEML
13. 13
Memory management for DL accelerators
▪ RAINBOW tool
▪ Different types of memory buffer
strategies
▪ Different types of optimizers
▪ Layer-by-layer on chip memory analysis
▪ Requirements determine best
heterogeneous execution plan
(combination of different strategies)
14. 14
▪ Common environment for running distributed applications
▪ WebAssembly runtime + Trusted Execution Environment
▪ Security for edge (and cloud) devices
▪ Advances on attestation
▪ Better support for edge devices
▪ Distributed (Byzantine fault-tolerant) attestation and configuration service
▪ Secure IoT Gateway
Security
15. 15
Simulation platform for ML
accelerators
▪ RISC-V SoCs and Custom
Function Units
▪ Improve test and
verification
▪ Co-simulate Verilog blocks
▪ Used in Google’s CFU
Playground
▪ Continuous integration
based in Gitlab and Google
Cloud Platform
Safety and Robustness
Robustness verification on DL models
▪ Tuning hyperparameters
16. 16
A compositional architecture framework for AIoT
Knowledge creation (e.g.
definition of safety goals).
Concept design (e.g.
introduction of redundancy
to fulfil safety goals).
Final design (e.g. assigning
functions to independent
processors to guarantee
redundancy).
Monitoring concept definition
(e.g. monitoring fulfilment of
safety goals at run-time).
Solution
Space
Problem
Space
17. 17
▪ Focus on collision detection/avoidance scenario
▪ Improve performance/cost ratio – AI processing hardware
distributed over the entire chain
Use case: Automotive
Challenge:
Distribution
of work
18. 18
▪ Control applications need DL-based condition classification
▪ On the edge device for low power consumption
▪ Suggestions for control and maintenance
▪ DL methods on all communication layers
▪ DL in a distributed architecture
▪ Dynamically configured systems
▪ Sensored testbench with 2 motors
▪ Acceleration, Magnetic field, Temperature,
IR-Cam (temperature), Current-Sensors, Torque
Use case: Industrial IoT – drive condition classification
▪ On / Off detection without
motor current or voltage
▪ Cooling fault detection
▪ Bearing fault detection
Challenge:
Low-power /
Efficiency
19. 19
Use case: Industrial IoT – Arc detection
▪ AI based pattern recognition for different local sensor data
▪ current, magnetic field, vibration, temperature, low resolution infrared picture
▪ Safety critical nature
▪ response time should be <10ms
▪ AI based or AI supported decision made by the sensor node itself or by a local part of the sensor
network
Challenge:
Accuracy
20. 20
▪ Face recognition
▪ Mobilenet SSD trained on WIDERFACE dataset
▪ Object detection
▪ YoloV3, Efficient-Net, yoloV4-tiny
▪ Gesture detection
▪ YoloV4-tiny with 3 Yolo layers (usually: 2 layers)
▪ Speech recognition
▪ Mozilla DeepSpeech
▪ AI Art: Style-Gan trained on works of arts
▪ Collect usage data in situation memory
Use case: Smart Mirror – Neural Networks
Challenge:
Data privacy,
Efficiency
21. 21
Thank you for your
attention.
Contact
Jens Hagemeyer, Carola Haumann
Bielefeld University, Germany
chaumann@cor-lab.uni-bielefeld.de
jhagemey@cit-ec.uni-bielefeld.de
22. 22
Bielefeld University (UNIBI) - Coordinator
Christmann (CHR)
University of Osnabrück (UOS)
Siemens (SIEMENS)
University of Neuchâtel (UNINE)
University of Lisbon (FC.ID)
Chalmers (CHALMERS)
University of Gothenburg (UGOT)
RISE (RISE)
EmbeDL (EMBEDL)
Veoneer (VEONEER)
Antmicro (ANT)
Partners
23. 23
▪ Increase safety, health and well being of residents – acceleration of AI
methods for demand-oriented user-home interaction
▪ Smart Mirror as central user interface
▪ Own mirror image can be seen normally
▪ Intuitive control over gesture and voice
▪ Shows personalized information
▪ Data privacy as the highest priority
▪ Edge computation of many neural networks
Use case: Smart Home / Assisted Living
24. 24
VEDLIoT Deep Learning Plattforms
Supported Computer-On-Module form factors
Raspberry Pi Compute
Module 4
Jetson Xavier NX
SMARC
Xilinx Kria
Jetson AGX Xavier
COM Express
(Type 6/7)
COM-HPC
Client (Type A-C)
COM-HPC
Server (Type D/E)
Size
(higher distance
is smaller)
I/O
Flexibility
Performance
Supported
Architectures
Market
Share
uRECS
RECS|Box
&
t.RECS
25. 25
Benchmark performance of DL accelerators
Comparison based on currently available architectures
VEDLIoT will include new specialized accelerators
0
50
100
150
200
250
300
350
Coral (M.2) Coral (Dev.) Xavier AGX
(LP)
Xavier AGX
(HP)
Xavier NX TX2 Nano GTX1660 ZU15 ZU3 Xeon-D1577 Epyc3451 Myriad GAP8
Energy Efficiency [GOPS/W]
ResNet50 Int 8 ResNet50 FP16 ResNet50 FP32
YoloV4 Int 8 YoloV4 FP16 YoloV4 FP32
MobileNet Int 8 MobileNet FP16 MobileNet FP32
26. 26
Flexible Accelerators for Deep Learning
DL
Model
DL Model
CPU, GPU-
SoC,
ML-SoC
FPGA-SoC
End of Moore’s law & dark silicon
=> Domain Specific Architectures (DSA)
Efficient, flexible, scalable accelerators for
the compute continuum
Algotecture
Optimized DL algorithms
Optimized toolchain
Optimized computer architecture
Heterogeneous DL
Accelerator
Algotecture/
Co-Designed DL
Accelerator
Compiler
Co-Design