This AI Performs Super Resolution in Less Than a Second

Share this video on

What's Hot

What's New

Top Grossing

Top of the Chart

Recommend

Moby Motion : Unbelievable. Fast computation is crucial for this technology to see mainstream use - especially since lots of people might want to use it on videos and get rid of 480p once and for all

kastakan : Finally we'll be able to watch Japanese porn! :D

Ville Pakarinen : I love how he always calls me a "fellow scholar" even though I don't do shit

Krishna Mohan : Ok. Time to stop buying 1000$ phones. Thanks AI. Just don't kill me in the future.

Catalepsy : Like those tv shows and movies "Enhance that" 🤣

phreakinpher : ENHANCE!

Menyhért Márton : What happens when you overdo the up-scaling? I'm curious if these algorithms produce distorted or unrealistic images.

CookingAndJava : Finally the police will actually be able to enhance low resolution surveillance camera footage, like in the movies!

Bob Salita : GitHub: https://github.com/fperazzi/proSR

sabo fx : *N O F U C K I N G W A Y* !!! This is really amazing! Cannot wait for the photoshop filter! Thanx m8!

Lurker1979 : The print industry will love this. Heck as a photographer I will love this option. Instead of having to cram more and more megapixels on to a chip and making more artifacts. Why not do this instead. Upscale a 20 megapixel image to 50 instead. I could even see this being applied to video. How cool would it be to have a old 60s sitcom that was filmed with TV cameras and has no film and upscale it to full HD.

21EC : oh wow....this in realtime could be used for an old game like Doom...the game is really pixelated, so it could be pretty cool to play it with higher res' graphics.

M Siemons : As a physicist let me remark: the upsampling and making up new pixels is very nice, it is NOT super resolution. Resolution of an image is very well defined in terms of the diffraction limit (the largest spatial frequency transmitted by your imaging system) and there are ways to overcome thus limit with super resolution imaging techniques (often in microscopy). AI does not improve this resolution, but should be seen as a smart interpolation of data/pixels.

Shall NotWither : This will definitely create legal problems for the photo industry and copyright in the short term as removing watermarks and enhancing images. Not to mention a future when you can download a low resolution video and then upscale it to HD while enjoying copyright free images because the AI only approximated the video. Amazing.

Jordan Scarrott : When you quote the time it takes to process these images, do these times not depend on the hardware they're running on? Is there an industry standard for this in this field? I don't know what to make of these times. I love the videos by the way. Really quality stuff

Reavenk : Great results! The intermediary results look like bilateral blur but then it gets really sharp and detailed really fast. There's some bilaterial blurish areas in some of the final iterative results, but you have to really be looking for it.

movax20h : Wow. Amazing technique. Almost magic. It puts details in the image that were not there before. The aligator picture ( 2:22 ) is amazing. You can see that the higher resolution picture has amazing detailed textures that were just smooth color with just few pixels and no texture at all. The algorithm learned what the texture should be based on other surranding features. Magic. There are some minor artifacts in one scene, at 1:41 , the landscape photo of a Greek(?) town with sea - the rocks in bottom right corner gets some additional high frequency texture, which doesn't feel natural. It can be also used as a possibly better subsitute for deconvolutional algorithms, to remove bluring, i.e. due to inaccurate focus. Greate video. Would be nice to mention how it actually works. I guess it is a convolutional deep neural network, so 5 second slide with architecture of the network would be useful.

skierpage : I don't think the AI uses related images to supersample, paper says "All presented models are trained with the DIV2K [34] training set, which contains 800 high-resolution images." If that set includes lots of alligator pictures then the results seem unfair 🙂. But what if the neural network recognize the kind of source picture and then supersampled based on related high-res images? Find higher-resolution pictures of fox fur or alligator skin or that particular village and supersample from training based off that.

Riyad : amazing, and all this time we were making fun of csi:miami enhance that licence plate scenes.

lIIlIllIlIl : You can download YouTube videos in 144p and watch in 4K

Dariush MJ : Like if you watched in 144p

John Doe : Enhance 224 to 176. Enhance. Stop. Move in. Stop. Pull out, track right, stop. Center and pull back. Stop. Track 45 right. Stop. Center and stop. Enhance 34 to 36. Pan right and pull back. Stop. Enhance 34 to 46. Pull back. Wait a minute. Go right. Stop. Enhance 57-19. Track 45 left. Stop. Enhance 15 to 23. Give me a hard copy right there.

Facebotter : Can't wait to see a similar technique used with audio. We'll be able to watch full-fledged movies from the past as if they were filmed today!

Nick Fotopoulos : I can't wait to use this on my old photos, videos, and movies!

21EC : damn....thats beyond crazy ! its like magic. but when do you think we will see this being used in for example google images...?

Anton Johansson : Did you have any success on images that werent first down sampled? It (quite reasonably) doesn't like JPEG.

21EC : perhaps this could also be used in video games...I mean in the future when theyll make this work in realtime

Lagraig O'Moof : Up to 8 times? Doesn't JPEG use 8x8 blocks within an image? This means that an 8x downscaled JPEG file ought to be equivalent to an 8x downscaled version of the original raw image, and so JPEGs would be ideal to train a network like this. i.e. Downscale JPEG 8x as raw pixels, upscale to 8x with the network as a raw image, apply JPEG compression to the output and then compare the 8x8 blocks between the original JPEG and the output.

dkwroot : This could be awesome if it gets implemented in 3D rendering. A scene could be rendered using 50% resolution and then scaled up in real time using super resolution.

Neoshaman Fulgurant : Next step, do it under .2ms on a tegra GPU, better a Mali 400mp. Then gaming is changed forever. Next step, take a low quality render turn it into cg.

Aadit Doshi : Looks like CSI was just way ahead of its time. ENHANCE!

Alex4Lolz : Are there any practical projects like a website or software that uses this except for "Let's enhance". I've found their results to be pretty lack luster.

Mister Niko : put this into Unity and Unreal Engines plus enable full raytracing = real time raytracing and most realistic graphics!

HebaruSan : I hope someday that we will be able to feed into an AI the first two Star Wars movies and have it output better prequels.

Cynosure : I watched it in 144p and was wondering why I couldn't see the differences

Squares : I'd like to see the result on a very low quality image coming from the crop of street webcam ! Like in the movies :)

Bazzralic : *THE MATRIX IS READY, HUMANS*

lIIlIllIlIl : I have a quick idea, I'll write it down here before I forget. What if you overfit a NN on some sort of data on a server side and than send low quality of that data along with that NN to client, so the NN can 'unzip' the data on the client side? For example you can overfit a NN on a HQ video and then send same video in low quality along with NN to a client, and then the NN interpolates the LQ video to HQ on the client side. This would make streaming much faster, given that the NN weighs less than the original HQ video.

Armuotas : Astronomers will be tripping about this! (Exo)planet and asteroid surface mapping, galaxy deep fields, proplids, etc.

Max Musterman : thx a lot for making all of those wonderful videos and i will watch every video from you. But it feels like you focus a lot on the visual/grafics area of ai. But i think there are a lot more fields in wich ai is used, in wich i am also very interested. love you whatever you focus on. would also love recommendations of other two minute paper-like ressources. Never forget: what an awesome time to be alife!

Name : Isn't nvidia's DLSS basically this but in real time?

André Pfitzner : When the processing power allows it to be integrated in super resolution TVs they will sell more.

Jack Sparrow : need 12GB GPU to upscale 520x520 picture 2x. So "1 sec" 30fps video, takes 24s -> 1 min takes 24 mins.

LanYarD : 2:47 BOI

eternalchao11 : A.I. painter Nice its about time.

William Taylor : it's weird why it's crooked, it's like they were shaking the camera. Bars aren't left then right then left, go, oh, bars are straight. surely, even shaking the camera, the shutter speed doesn't matter, all the light hit at the same time. first you have implied hue direction, then gamut falloff, that's four times. I gotta think of some spooky shadow detection for the bars trick. Your really looking for a three standard deviation improvement. So you got green 0 to 255, now 40 would be way to noticable. it might be another object, but 120 has to be the same object. So you got any 9 points with RGB. The fox has no improvement, but I can see how each whispy tuft should have a solid grey, 124,124,124. This is where gamut augmentation comes in. You can sharply take the 5 point parabolas' and add them together. But now Mr camera shaker let in 3d light.

John Leonard : Mind blowing

Wild Animal Channel : But does it create fake images? So shouldn't be used for news reports. Or it would be fake news!

pabanoid : psx pre-rendered background upscaling :))))

goosenp : Is the execution time measured for running the inference on the GPU or CPU?