AA coloring issues

This section is dedicated to the feature & improvement requests (be sure what you are asking does not exist yet in TVPaint Animation ;) )
Post Reply
User avatar
Peter Wassink
Posts: 4436
Joined: 17 Feb 2006, 15:38
Location: Amsterdam
Contact:

AA coloring issues

Post by Peter Wassink »

Precise coloring with Anti-Alias can be problematic
with the paintbucket set to anti alias we get a 'balooning'effect on the fill

this also happens when exporting the C&T layer set to anti-alias


This Anti-Aliasing effect seems to me to be a bug, or a symptom of some non optimized algorythm.
It was reported earlier i believe, cant relocate that but ran into this issue again.

The fill tool with AA or the Col&Texture layer set to extract with AA, both give the same distortion of the filled shape,
making it look as if its blown up.

i would think that a perfect square should look the same with or without AA,
only when a line has an angle other then 0° 90° 270° or 360° should AA kick in?
to me it seems weird that the AA would cause a square to blow up like a balloon even growing outside of its fill boundaries.





Another thing i noticed with is that the fill border goes up to the border on top and Left of the fill.
But it goes over 1 pixel to right and bottom boundary?
Why is the fill asymmetrical?
You can see it in this image, where i put the layer with black boundaries at 30% opacity

colo&texture-AA-issue.png
colo&texture-AA-issue.png (16.04 KiB) Viewed 16217 times
Peter Wassink - 2D animator
• PC: Win11/64 Pro - AMD Ryzen 9 5900X 12-Core - 64Gb RAM
• laptop: Win10/64 Pro - i7-4600@2.1 GHz - 16Gb RAM
Xavier
Posts: 1855
Joined: 01 Oct 2019, 11:08

Re: AA coloring issues

Post by Xavier »

Yes, i think you reported it in the beta thread. Thanks anyway, it's better here :)
User avatar
Peter Wassink
Posts: 4436
Joined: 17 Feb 2006, 15:38
Location: Amsterdam
Contact:

Re: AA coloring issues

Post by Peter Wassink »

yes we are currently struggling with this issue.
And when i revisited my post, i saw you suggested i put it here.

i mean this a-symmetrical filling.
we use the C&T and these edges cause us problems in compositing.
colo&texture-AA-issue2 .png
colo&texture-AA-issue2 .png (12.49 KiB) Viewed 16186 times
Last edited by Peter Wassink on 25 Nov 2020, 09:47, edited 4 times in total.
Peter Wassink - 2D animator
• PC: Win11/64 Pro - AMD Ryzen 9 5900X 12-Core - 64Gb RAM
• laptop: Win10/64 Pro - i7-4600@2.1 GHz - 16Gb RAM
Xavier
Posts: 1855
Joined: 01 Oct 2019, 11:08

Re: AA coloring issues

Post by Xavier »

The multiply blending mode is a pain to do. I also have an aliasing issue...
User avatar
Peter Wassink
Posts: 4436
Joined: 17 Feb 2006, 15:38
Location: Amsterdam
Contact:

Re: AA coloring issues

Post by Peter Wassink »

Xavier wrote: 25 Nov 2020, 09:39 The multiply blending mode is a pain to do. I also have an aliasing issue...
Yes i imagine Aliasing is hard to get right
i'm not sure what multiply has to do with this issue specificly, could you elaborate?
Peter Wassink - 2D animator
• PC: Win11/64 Pro - AMD Ryzen 9 5900X 12-Core - 64Gb RAM
• laptop: Win10/64 Pro - i7-4600@2.1 GHz - 16Gb RAM
Xavier
Posts: 1855
Joined: 01 Oct 2019, 11:08

Re: AA coloring issues

Post by Xavier »

Peter Wassink wrote: 25 Nov 2020, 09:49 i'm not sure what multiply has to do with this issue specificly, could you elaborate?
Completely unrelated, just to let you know on which topic I am currently :)
Xavier
Posts: 1855
Joined: 01 Oct 2019, 11:08

Re: AA coloring issues

Post by Xavier »

Peter Wassink wrote: 25 Nov 2020, 09:36 yes we are currently struggling with this issue.
And when i revisited my post, i saw you suggested i put it here.

i mean this a-symmetrical filling.
we use the C&T and these edges cause us problems in compositing.
colo&texture-AA-issue2 .png
Thanks for the detail level, no worries I'll have a look.
User avatar
artfx
Posts: 676
Joined: 20 Sep 2006, 18:52
Location: Shanghai, China
Contact:

Re: AA coloring issues

Post by artfx »

I wonder if this is why I notice in videos about Japanese studios (that use TVPaint) that they use no antialiasing, on their lines or colors... none... nowhere.

Back in the day when we had a 2D team at the studio in Shanghai, I don't know what software they were using for ink and paint, but I was surprised to see the same thing. No antialiasing on anything.

I wonder what resolution they must work at to make that look good.
Terrence Walker
MacBook Pro (Retina, 13-inch, Early 2015)
MacOS Monterey Version 12.6.2
2.7 GHz Intel Core i5, 8 GB 1867 MHz DDR3, Intel Iris Graphics 6100
User avatar
Hironori Takagi
Posts: 313
Joined: 14 May 2018, 10:15
Location: Tokyo, Japan
Contact:

Re: AA coloring issues

Post by Hironori Takagi »

artfx wrote: 21 Dec 2020, 22:19 I wonder if this is why I notice in videos about Japanese studios (that use TVPaint) that they use no antialiasing, on their lines or colors... none... nowhere.

Back in the day when we had a 2D team at the studio in Shanghai, I don't know what software they were using for ink and paint, but I was surprised to see the same thing. No antialiasing on anything.

I wonder what resolution they must work at to make that look good.
There are two reasons why we don't color with antialiasing.
(1) It takes time to color.
(2) When compositing, use the color area as mask.
After adding textures and effects with each color area, antialiasing is handled with plug-in effects.
TVPaint 11.7.2(Nov 14 2023), Windows11 Pro, HP Spectre x360 Convertible 14-ea0xxx / TVPaint 11.7.1(Dec 22 2022) Mac OS 11.6, Apple MacBookPro M1 2020
User avatar
artfx
Posts: 676
Joined: 20 Sep 2006, 18:52
Location: Shanghai, China
Contact:

Re: AA coloring issues

Post by artfx »

Hironori Takagi wrote: 24 Dec 2020, 09:37
artfx wrote: 21 Dec 2020, 22:19 I wonder if this is why I notice in videos about Japanese studios (that use TVPaint) that they use no antialiasing, on their lines or colors... none... nowhere.

Back in the day when we had a 2D team at the studio in Shanghai, I don't know what software they were using for ink and paint, but I was surprised to see the same thing. No antialiasing on anything.

I wonder what resolution they must work at to make that look good.
There are two reasons why we don't color with antialiasing.
(1) It takes time to color.
(2) When compositing, use the color area as mask.
After adding textures and effects with each color area, antialiasing is handled with plug-in effects.
Wow! Thanks for answering and explaining that. I didn't know that antialiasing could be added later. This is very good information.
Terrence Walker
MacBook Pro (Retina, 13-inch, Early 2015)
MacOS Monterey Version 12.6.2
2.7 GHz Intel Core i5, 8 GB 1867 MHz DDR3, Intel Iris Graphics 6100
User avatar
D.T. Nethery
Posts: 4225
Joined: 27 Sep 2006, 19:19

Re: AA coloring issues

Post by D.T. Nethery »

Hironori Takagi wrote: 24 Dec 2020, 09:37 There are two reasons why we don't color with antialiasing.
(1) It takes time to color.

(2) When compositing, use the color area as mask.

After adding textures and effects with each color area, antialiasing is handled with plug-in effects.
.

Are the plug-ins to add antialiasing to the image plug-ins for TVPaint or in another application ?

Animator, TVPaint Beta-Tester, Animation Educator and Consultant.
MacOS 12.7.1 Monterey , Mac Mini (2018) , 3.2 GHz 6-Core Intel Core i7,
16 GB RAM , TVPaint PRO 11.7.1 - 64bit , Wacom Cintiq 21UX 2nd Gen.
,Wacom Intuos Pro 5 , Wacom driver version 6.3.39-1
User avatar
Hironori Takagi
Posts: 313
Joined: 14 May 2018, 10:15
Location: Tokyo, Japan
Contact:

Re: AA coloring issues

Post by Hironori Takagi »

D.T. Nethery wrote: 28 Dec 2020, 20:31
Hironori Takagi wrote: 24 Dec 2020, 09:37 There are two reasons why we don't color with antialiasing.
(1) It takes time to color.

(2) When compositing, use the color area as mask.

After adding textures and effects with each color area, antialiasing is handled with plug-in effects.
.

Are the plug-ins to add antialiasing to the image plug-ins for TVPaint or in another application ?
For example, we use After Effects plug-in "OLM Smoother".
https://olm.co.jp/rd/technology/tools/?lang=en
TVPaint 11.7.2(Nov 14 2023), Windows11 Pro, HP Spectre x360 Convertible 14-ea0xxx / TVPaint 11.7.1(Dec 22 2022) Mac OS 11.6, Apple MacBookPro M1 2020
User avatar
D.T. Nethery
Posts: 4225
Joined: 27 Sep 2006, 19:19

Re: AA coloring issues

Post by D.T. Nethery »

Hironori Takagi wrote: 29 Dec 2020, 10:37
D.T. Nethery wrote: 28 Dec 2020, 20:31
Are the plug-ins to add antialiasing to the image plug-ins for TVPaint or in another application ?
For example, we use After Effects plug-in "OLM Smoother".
https://olm.co.jp/rd/technology/tools/?lang=en

Thank you. I've downloaded it and will try it.

Do you apply the OLM Smoother to a single composite layer (characters and backgrounds) in After Effects or is OLM Smoother applied to
separate layers of characters and backgrounds in AE before doing the final export to .MOV or .MP4 from Adobe Media Encoder ?

Animator, TVPaint Beta-Tester, Animation Educator and Consultant.
MacOS 12.7.1 Monterey , Mac Mini (2018) , 3.2 GHz 6-Core Intel Core i7,
16 GB RAM , TVPaint PRO 11.7.1 - 64bit , Wacom Cintiq 21UX 2nd Gen.
,Wacom Intuos Pro 5 , Wacom driver version 6.3.39-1
Post Reply