01:24PM EST – After yesterday announcing both the Snapdragon 765 and Snapdragon 865 mobile platforms, which will aim to provide a lot of 5G coverage to 2020 devices, Qualcomm today is going to go deep into what is inside both of these chips. We’re here will full live blog coverage – the talk starts at 7pm UTC / 2pm ET.
02:02PM EST – Alex Katouzian to the stage
02:03PM EST – 560k views on the live stream yesterday, more than all of last year combined
02:03PM EST – Yesterday, made another 5G standalone data call with Ericsson
02:03PM EST – Demo areas are open this afternoon with S865 and S765
02:04PM EST – Keith Kressen to the stage, SVP Product Management
02:05PM EST – First Snapdragon summit at this venue was 2017. AI won at Go, Bitcoin was $17k
02:07PM EST – Why 2017? Q4 2017 was when Qualcomm finalized the definition of the S865
02:08PM EST – 865 was prepared for the 2019 world back in 2017
02:08PM EST – 144 Hz displays, 200 MP cameras, ultrasonic fingerprint
02:09PM EST – Christopher Patrick to the stage, SVP Engineering
02:10PM EST – Back in 2013, Snapdragon 800 was launched
02:10PM EST – Lot of industry firsts
02:10PM EST – first ever 4K capture/playback
02:10PM EST – first ever LTE with CA
02:12PM EST – 1st year of development is IP definition
02:13PM EST – Algorithm in hardware definition
02:13PM EST – ML techniques that require DSP adjustments all done in that first year
02:14PM EST – also fundamental things like transistor definition and packaging technology
02:15PM EST – After the SoC, have to consider the device and the other components. Modem, power amplifiers, RF, audio, etc
02:15PM EST – e.g. 8K, need to make sure power and energy at the right parts of the system to also have minimum consumption on the battery
02:16PM EST – also thermal management
02:16PM EST – Packaging, security
02:16PM EST – Then onto handset and ecosystem
02:17PM EST – On Year 3 of development, finalizing the design in hardware, software, and optimization
02:18PM EST – (This means that Qualcomm is likely working on 885 or even 895 today, ready for 2021/2022)
02:19PM EST – 10k engineers to bring S865 to life
02:19PM EST – Keith Kressin back to the stage
02:20PM EST – First up, the X55 modem
02:20PM EST – the modem, the RF and the front end was all developed as a single system
02:20PM EST – 2nd gen 5G modem to antenna solution
02:20PM EST – It’s complex, so you need to define it as a system
02:20PM EST – Sub 6 GHz and mmWave support
02:21PM EST – Support for Dynamic Spectrum Sharing
02:21PM EST – Support Carrier Aggregation
02:21PM EST – Premium devices have to be future proofed
02:22PM EST – Wi-Fi 6 through FastConnect 6800
02:22PM EST – 1.8 Gbps with simultaneous 2.4G and 5G band operation
02:22PM EST – First to support 8-way uplink MU-MIMO
02:22PM EST – BT5.1 and Aptex
02:23PM EST – Support two wireless stereo plus for airbuds
02:23PM EST – can connect to two sets of airbuds independently with 2x range
02:23PM EST – S865 built on TSMC 7NP
02:23PM EST – 765 and 765G built on Samsung 7nm EUV
02:24PM EST – Spectra 480 CV-ISP
02:24PM EST – 2 gigapixel/sec
02:24PM EST – redesigned for higher resolution capture and lower power
02:24PM EST – 4K video 64MP pictures at 30 fps
02:24PM EST – 4x pixels per clock
02:24PM EST – EVA new video engine
02:25PM EST – redesigned the pipeline for lower power, not simply upping the freq
02:25PM EST – library for object tracking, 25% lower latency and 50% lower power than S855
02:25PM EST – HEIF support
02:25PM EST – Adreno 650
02:26PM EST – Most efficient graphics in the world
02:26PM EST – Goal to be the best sustained in performance
02:26PM EST – Not to be #1 in peak for benchmarks – but for the best performance after 30 minutes of gaming
02:26PM EST – Performance in Alaska has to be the same in Maui
02:26PM EST – 25% faster rendering, 35% more power efficient
02:27PM EST – Optimized the design point of GPU for 90 fps
02:27PM EST – 35% better power eff at 90 fps
02:27PM EST – Display processing engine has local tone mapping for gaming
02:27PM EST – Also software for graphics
02:27PM EST – CPU time
02:28PM EST – 1x A77 at 2.84 G
02:28PM EST – 3 x A77 at 2.4 GHz
02:28PM EST – 4 x A55 at 1.8 GHz
02:28PM EST – local L2 caches, all shared L3
02:28PM EST – S855 had 2MB L3, this one has 4MB of L3
02:29PM EST – Hexagon to support new tensor accelerator, 4x more perf
02:29PM EST – Also 35% more power efficient
02:29PM EST – 4 threads of scalar processing
02:30PM EST – 15 TOPs when using CPU+GPU+DSP
02:30PM EST – LPDDR5-5500 support (2750 MHz)
02:30PM EST – 3MB System cache
02:30PM EST – Support for LP4 or LP5, up to the OEM based on commodity pricing
02:31PM EST – Sensing hub
02:31PM EST – sub 1mW camera wake up, sub 1mA, scalable framework
02:32PM EST – Offering a separate chip for that sub 1mW camera – if OEMs want an always on camera
02:32PM EST – hands up who wants an always on camera on their phone
02:32PM EST – Smart APIs
02:33PM EST – Multiple wake up words
02:33PM EST – Security
02:34PM EST – Jesse Seed, Senior Director of Security to the stage
02:34PM EST – If you don’t think about your data, Qualcomm does
02:35PM EST – Protected pipelines for secure data
02:35PM EST – Cannot copy data out of the secure region
02:35PM EST – even when malware is installed
02:36PM EST – Hardware supported Attestation for banks
02:36PM EST – There’s no standard for something like face authentication
02:37PM EST – ‘Moving at the speed of snapdragon’
02:37PM EST – isolated secure processing unit for assured processing environments
02:37PM EST – was first on S855
02:38PM EST – used to be only provided by external chips
02:38PM EST – S865 will be first dual SIM dual standby SoC, to replace dual SIMs or dual eSIMs
02:38PM EST – Work with Google on Android Security
02:38PM EST – el goog to the stage
02:39PM EST – research and security. Qualcomm was the first to support the new secure credential API in Android
02:39PM EST – Control flow integrity
02:40PM EST – Enabling hardware security through software as well
02:40PM EST – Security needs to be seemless and unintrusive
02:41PM EST – Qualcomm is suggesting that you can have core ID on your phone. Drivers ID etc
02:41PM EST – (this will not end well)
02:41PM EST – ‘Security is glamorous’
02:42PM EST – More on AI, QC VP AI to the stage
02:43PM EST – AI use cases
02:43PM EST – Add more and more intelligence to devices
02:43PM EST – Understand their surroundings
02:43PM EST – More than 1 billion devices enabled that use QC AI tech
02:44PM EST – Different AI use cases – music detection
02:44PM EST – doing it with on-device AI rather than doing it in the cloud
02:44PM EST – always on neural nets at ultra-low power
02:44PM EST – on-device speech to text
02:45PM EST – bypassing the cloud
02:45PM EST – inference happens on the device instead. better experience, lower latency
02:45PM EST – (I thought the low latency of 5G would mean that wouldn’t matter? Isn’t it more power efficient in the cloud?)
02:46PM EST – ‘AI is happening as we speak’
02:46PM EST – Transcription and translation in real time on the device
02:47PM EST – Already running on S865 on stage
02:47PM EST – this is the beast mode for AI
02:47PM EST – Three tools: software, hardware, and dev tools
02:48PM EST – AI is already being used on almost every segment of the mobile platform (all the blue segments)
02:48PM EST – e.g. AI algorithms in the modem to help improve antenna perf
02:48PM EST – 5gh AI Engine
02:48PM EST – Adreno 650 does 2x TOPs, supports FP16 and FP32
02:49PM EST – Hexagon 698 uses deep learning bandwidth compression
02:49PM EST – 50% lossless compression
02:49PM EST – Frees up bandwidth, saves power
02:49PM EST – LPDDR5 has 30% more bandwidth
02:50PM EST – Everything is a tensor
02:50PM EST – Different parts work best are scalar/vector/tensor
02:50PM EST – Others find this difficult to manage – seemless on QC
02:51PM EST – S845 was 3 TOPs, S855 was 7 TOPs, S865 is 15 TOPs
02:51PM EST – Note that’s CPU + GPU + DSP. You aren’t going to use all three at once, ever.
02:52PM EST – peak perf and peak perf/watt
02:53PM EST – Now sensing hub
02:53PM EST – At low power sub 1mA, device can tell if the audio is the right person
02:53PM EST – Device knows when you are walking, or in the car, all at low power
02:54PM EST – Now AI software
02:55PM EST – It’s already in your device, and 865 makes it better
02:55PM EST – Software stack and 25+ partners
02:55PM EST – 25 partners and apps is at the top of the stack
02:56PM EST – partners in the audience
02:56PM EST – Framework support is pytorch, tensorflow, onnx
02:56PM EST – Onnx is the interchange format for other frameworks not listed
02:57PM EST – Qualcomm wants the least amount of friction to enable AI on QC hardware
02:57PM EST – Qualcomm Neural Processing SDK
02:57PM EST – (note I believe you still have to be a preferred partner to get access to the frameworks that do API calls to the DSP)
02:58PM EST – NNAPI support, obviously. It’s just a shame that device OEMs can’t use the latest drivers to enable it
02:58PM EST – Power, perf, access, features
02:58PM EST – best in class solution
02:59PM EST – QC AI research into things like data-free quantization
02:59PM EST – Accelerate all the key use cases
02:59PM EST – 3-5x speed ups on Hexagon
03:00PM EST – Moving automatic speech recognition with Google, moving it from CPU to hexagon on S865, gives 3x power savings
03:01PM EST – 160+ operator support for AI
03:01PM EST – operator = math functions in AI, not wireless operator
03:01PM EST – User defined operators now on the platform
03:02PM EST – The power of snapdragon powered by you
03:02PM EST – Tensorflow lite
03:03PM EST – Hexagon NN Direct for Tensflow Lite and more
03:04PM EST – Snap Inc to the stage
03:05PM EST – Creative tools on Snapchat using AI
03:06PM EST – only 23 ms per frame – smooth 30 FPS on S865
03:06PM EST – Using hexagon direct
03:07PM EST – back to Qualcomm
03:08PM EST – Qualcomm AI Model Efficiency Tookit – 3x compression with 1% lower accuracy
03:09PM EST – FP32 to INT8
03:13PM EST – Intelligence at all parts of the network: 5G and AI
03:13PM EST – Multiple 320p streams to a single device, upscaled to 720p on the device, saves data on the device
03:14PM EST – tiktok super resolution
03:15PM EST – Loom.ai to the stage. 3D avatars
03:17PM EST – face tracking, avatar application, and background adjustment
03:17PM EST – For meetings etc
03:17PM EST – uses Hexagon
03:17PM EST – avatar to avatar calls at lower data rate
03:18PM EST – QC back to the stage
03:18PM EST – loom.ai uses four different CNNs on the hexagon
03:18PM EST – Also Gaming, XR, Contextual awareness
03:19PM EST – textures in a game adjust based on facial detection of player mood
03:19PM EST – for example
03:20PM EST – You need to offer a complete AI solution: hardware, software, and dev tools
03:20PM EST – Now camera
03:21PM EST – Smartphones enable more camera features than a ‘single camera’
03:21PM EST – Computational photography
03:23PM EST – 5 cameras per person in a household, due to smartphone cameras
03:23PM EST – Turning the world into photographers
03:24PM EST – Qualcomm powers the most smartphone cameras on the planet
03:25PM EST – Step forward on Spectra 480 ISP. Biggest update ever
03:26PM EST – 2 gigapixels per second
03:26PM EST – 64MP at 30 fps
03:27PM EST – pixels matter more than raw GB/s
03:27PM EST – 4x pixel processing
03:28PM EST – 4 pixels per clock. This is QC matching the competition
03:29PM EST – 3 pieces to the ISP
03:29PM EST – Capture, video analytics, and HEIF
03:30PM EST – With the margin and the ISP, can use 9x more autofocus points
03:30PM EST – Quad CFA image sensors coming
03:31PM EST – looks like some form of auto pixel fusion but not
03:31PM EST – S865 supports Quad CFA
03:31PM EST – Dedicated cores for noise reduction in the ISP
03:32PM EST – Also local contrast enhancement
03:32PM EST – 40% more pixel processing for noise reduction during video capture
03:33PM EST – S865 enables more simultaneous captures ever
03:33PM EST – Spectra 480 enables full 64MP picture capture during 4K video recording
03:33PM EST – assuming you have the sensors
03:34PM EST – 200MP camera capture support
03:34PM EST – QC has partnered with sensor makers, so 200MP handsets in 2020
03:35PM EST – 16% lower power video capture
03:35PM EST – at 4K60
03:35PM EST – ‘18% more texture in your photos’ – er what? how do you measure that?
03:35PM EST – HEIF
03:36PM EST – Depth map can be stored in the HEIF container
03:36PM EST – Enables post processing
03:37PM EST – (again, it requires the OEM to support capturing the data)
03:38PM EST – Spectra + AI
03:40PM EST – demo time
03:40PM EST – Semantic segmentation with AI
03:41PM EST – Applying algorithms with different strength integrities
03:42PM EST – Using AI to recoup details on normal filters
03:42PM EST – Applying the right algorithm at the right place
03:43PM EST – accelerated by S865
03:44PM EST – Modern cameras can’t smooth zoom between cameras – they don’t reproduce the same results because of discontinuities
03:45PM EST – *modern smartphones
03:45PM EST – This is a use for AI
03:46PM EST – 8K30 on S865
03:46PM EST – recording
03:46PM EST – 33MP per frame, * 30 fps, = 0.99 gigapixel/sec
03:46PM EST – (if it’s 2 gigapixel, why not 8K60 ??)
03:47PM EST – 4K HDR10+ support with S865 with Dolby support
03:48PM EST – Dolby to the stage
03:50PM EST – millions of Dolby devices, over 2500 movies and TV shows
03:52PM EST – QC back to the stage
03:53PM EST – 4K120 supported on S865
03:53PM EST – 120fps playback too with devices that support higher refresh rates
03:53PM EST – Slow mo in 4K
03:53PM EST – 4x 4K slowmo
03:54PM EST – slomo with no limits
03:54PM EST – native capturing at 720p960
03:54PM EST – no limits means no limit on video recording
03:55PM EST – so a 10 minute video recording at 960 fps
03:56PM EST – or a 10 hour video
03:58PM EST – Leilani DeLeon to the stage to talk gaming and XR
03:59PM EST – Snapdragon Elite Gaming
03:59PM EST – Beast Mode again
03:59PM EST – ‘gaming is 43% of all smartphone use, 74% of all app store spend
04:00PM EST – Gaming in XR
04:00PM EST – We are now in a mobile first generation
04:01PM EST – perhaps even a ‘mobile only’ generation
04:01PM EST – 518m eSports viewers in 2020 expected
04:01PM EST – (but mobile gaming eSports? that’s super low)
04:04PM EST – some eSports person to the stage
04:06PM EST – ‘I want 144 Hz on my phone for fortnite’
04:07PM EST – 80% of gamers in China want specialized gaming phones
04:08PM EST – because winning matters
04:09PM EST – Todd LeMoire to the stage, Principal Engineer, Gaming
04:10PM EST – Fortnite – seemless experience optimized for Qualcomm Elite Gaming
04:10PM EST – S865 pushes beyond the limits
04:11PM EST – Game Smoother
04:11PM EST – Jank Reducer at any refresh rate
04:11PM EST – Adaptive Game Performance engine – working at the sub-frame level
04:12PM EST – 90 FPS Gaming Mode coming to PUBG
04:12PM EST – 120 FPS with some tencent games
04:12PM EST – 1st 144 Hz diplays with S865
04:13PM EST – 10-bit HDR gaming for Android
04:14PM EST – Game Color Plus – enhancing quality on Adreno
04:14PM EST – No changes to the game code are necessary
04:14PM EST – Works on Lineage II Revolution
04:15PM EST – Doesn’t add latency
04:15PM EST – Boosts color saturation and contrast. More vivid scenes
04:15PM EST – Adreno HDR Fast Blend
04:17PM EST – Upgradeable mobile drivers !
04:17PM EST – This is important
04:18PM EST – Driver updates through the Play Store
04:18PM EST – Phone agnostic
04:18PM EST – Desktop Forward Rendering
04:19PM EST – Depth of Field through Desktop Forward Rendering
04:20PM EST – Multiple Dynamic Lights
04:20PM EST – Dynamic Shadows
04:22PM EST – One of the key aspects of desktop drivers is day 0 updates for new launches. Qualcomm hasn’t said anything about doing this for mobile. Yet
04:23PM EST – S865 summary time
04:25PM EST – All android gaming smartphones today are based on Snapdragon. 2020 will be the same
04:26PM EST – That’s a wrap for today. The demo room opens in a couple of hours, will report what’s there 🙂