ThinCI Q&A on Deep-Learning

　　ThinCI is among many startups that have sprung up in the past 18 months claiming breakthroughs in the realm of deep-learning processors.

　　The Eldorado, Calif.-based company, with a sizable team in India, appears confident that it can stand head and shoulders above its competitors. It is in the midst of taping out its first processor, and the company’s partnerships and business models are well advanced.

　　ThinCI (pronounced “Think-Eye”) will unveil next week at Hot Chips details of its high-performance processor, Graph Streaming Processor (GSP), billed as a “next-generation computing architecture.”

　　Meanwhile, Denso, a large Japanese tier one and a key investor in ThinCI, last week revealed that it has established a new subsidiary to design and develop semiconductor IP cores for key components necessary in automated driving. The architecture of a new chip is being jointly developed with ThinCI, Denso announced during a press conference in Japan. Denso calls it a Data Flow Processor (DFP), describing it as “very different from CPU or GPU.”

　　EE Times last week caught up with Dinakar Munagala, ThinCI’s CEO, and discussed his company’s latest developments and his view on the automotive/non-auto markets. Here’s an excerpt of our conversation.

　　EE Times: It’s been almost a year since EE Times talked to you. Where do you stand today with your chip development?

　　Mungala: We’re in the middle of taping out. Our first processor will be coming out by the end of Q3. Our chip architecture has been benchmarked through multiple engagements — by automotive and non-automotive companies, and it’s well received.

　　EE Times: Who’s your lead customer?

　　Mungala: It’s an automotive company, which we can’t name. We also have a few select customers in the other market segments.

　　EE Times: That lead customer you are referring to… is that Denso? Denso said its new subsidiary, called NSITEXE Inc., will license semiconductor IP cores optimized for in-vehicle applications to SoC manufacturers.

　　Mungala: Denso is our investor and partner. We have a development effort going on with Denso. However, that’s different from what we are doing here at ThinCI. I was referring to our own customers.

　　EE Times: So, are you saying that ThinCI won’t be doing the IP business?

　　Mungala: No, we’re not in the IP business. A lot of startups start with an IP business model because they can’t sell their own chips. But that’s not the case with us.

　　EE Times: So, what will you be selling?

　　Mungala: We’re initially focused on selling modules featuring our chips — like accelerator cards. Just as NVidia has succeeded in selling PCI-based GPU cards into servers and data centers, we will also launch PCI accelerator boards. The module would make it easy for our select customers to see the efficiency of our processor.

　　EE Times: Does that mean you are going after the data-center market?

　　Mungala: We can deploy our processor to both the cloud and the edge, but our focus is on edge computing. Our processor can be used as a co-processor to speed up data processing in the cloud. But it is ideally employed when placed much closer to where data is being generated — like right next to an ISP (image signal processor).

　　EE Times: So you are saying that both ThinCI’s technology and business models are scalable — from chips to modules, deep learning training to inference. And the processor’s applications are not limited to automotive…

　　Mungala: Yes. We think our processor can be used everywhere from sensors, vision and cameras, to microphones (speech), smart factor, smart retail, and semantic data analysis… because the processor can speed up the computation while reducing cost and power.

　　EE Times: Can you explain the architecture of your processor?

　　Mungala: Unfortunately, we can’t disclose it until Hot Chips next week.

　　EE Times: To paraphrase what you previously told us, your processor is based on “a massively parallel architecture designed to process multiple compute nodes of a task graph at the same time.” Correct?

　　Mungala: Yes. It’s also important to note that our processor does not need a huge batch of images to do inference, thus making it ideal for edge computing.

　　EE Times: What do you mean by that?

　　Mungala: The batch size relates to how efficiently a machine can process small amounts of data. Any inference engine can work on a single frame (image, sound sample, etc.). What’s different is how their various efficiencies arise when they get larger sets of data. For example, if you have a complex machine that “likes” to work on many things in parallel to achieve an economy of scale for power, then if you are only running one image through you can spend relatively too much power. This would be like an eight-lane highway with only one car at a time. The single car has no traffic and flows freely, but you’ve wasted seven lanes.

　　In the specifics of the inference engine, this relates to how data flows through the machine, and the overhead to manage the data. In the case of ThinCI, we attain similar efficiencies (measured in images/second/Watt) whether it’s a batch size of 1 (one picture) or batch size of 128 (lots of pictures). This is important to edge inference where latency matters, such as in automotive safety systems. At 30 frames/second on high resolution video it takes more than four seconds to collect 128 images. At 65 mph a car travels 370 feet, and that sort of “sampling interval” would be unacceptable for a collision prevention system.

　　EE Times: You mentioned that ThinCI’s processor can be used right next to an ISP, so that the processor can reduce the data that needs to be handed off to sensor fusion. But in reality, carmakers today are far from agreement when it comes to where the sensor fusion should take place inside a car.

　　Mungala: We’re aware of it. I see ThinCI’s processor — with a rich set of I/O interfaces (i.e. 4 MIPI interfaces) — can play more of a hybrid role. While it can used closer to the image signal processor, it can be also used as a central or “global” fusion chip — to be used in applications like path planning.

　　Again, whether ThinCI’s processor is used at the edge, hybrid, or in the cloud, the beauty of our solution is that you can use one set of software for all.

　　Denso's new subsidiary

　　While ThinCI declined to comment on specifics of Denso’s new subsidiary, NSITEXE, Tatsuya Takemoto, editor in chief of EE Times Japan, reported it as follows.

　　Asked about Denso’s DFP which is based on ThinCI’s processor architecture, NSITEXE’s president-to-be Yukihide Niimi explained during the press conference in Japan that the DFP will be responsible for “path finding.” He said, “DFP will co-exist with other chips — such as Nvidia’s GPU, Intel’s CPU and Toshiba’s Visconti (which Denso is collaborating).”

　　DFP is particularly agile and efficient in data processing, he added, as it streams data using extreme parallelism. “Through its efficient processing, the DFP can do instantaneous analysis and reflexive decision-making,” he noted. Another plus is that software running on the DFP can be programmed in C.

　　DFP development has been going smoothly, said Niimi. “As we are getting favorable evaluation on our technology, we are confident semiconductor companies will license our IP.”

　　Asked about the first DFP that will be integrated in commercial cars by the first half of 2020, Denso expects it to be “a version of DFP consisting of four cores.”

　　Other than ThinCI, partners for NSITEXE’s DFP development include ARM and Imagination. The collaborations with ARM include functional safety of DFP using ARM CPUs. NSITEX plans to collaborate with Imagination in applying multithreaded hardware expertise accumulated by MIPS to enable the parallelism of the DFP’s software.