1. SFT
  2. Archiwum
  3. Vol. 66
  4. Analiza cech jakości obrazu oraz arte...

Analiza cech jakości obrazu oraz artefaktów przetwarzania jako fundament detekcji deepfake

dr inż. Karol Jędrasiak

Pobierz artykuł
Open Access
DOI: 10.12845/sft.66.2.2025.10, strony: 168 – 201

Abstrakt

Cel: Celem artykułu była empiryczna weryfikacja hipotezy, że deskryptory jakości obrazu oraz artefaktów przetwarzania mogą stanowić stabilny i interpretowalny fundament detekcji deepfake w warunkach rzeczywistej dystrybucji (ang. in-the-wild). Badanie miało na celu zidentyfikowanie mierzalnych cech wizualnych, zakorzenionych w fizyce akwizycji i przetwarzania sygnału, które pozwalają rozróżniać treści syntetyczne od autentycznych z wysoką odpornością na degradacje platformowe i manipulacje rekodujące.

Projekt i metody: Opracowano i wykorzystano zbiór danych DeepFake RealWorld (DFRW) obejmujący 46 371 klipów (4186 autentycznych i 42 185 syntetycznych), odzwierciedlający rzeczywiste łańcuchy przetwarzania i modele generacyjne (GAN, dyfuzja, reenactment, face swap). Dla każdego nagrania obliczono zestaw 20 deskryptorów jakości i artefaktów, w tym BRISQUE, NIQE, PIQE, BLIINDS II, V-BLIINDS, CPBD, Wang–Bovik, PRNU, CFA i markery podwójnej kompresji. Selekcję cech przeprowadzono bez klasyfikatorów, poprzez progowanie anomalii definiowanych na klasie rzeczywistej oraz obliczenie wskaźników p_df, p_real, Δp i PR z kontrolą FDR dla stabilności i odporności na degradacje platformowe.

Wyniki: Uzyskano istotne różnice między treściami syntetycznymi i autentycznymi: średnio p_df = 41,92%, p_real = 26,54%, Δp = 0,15, PR = 1,56. Najwyższą skuteczność i stabilność w detekcji deepfake wykazały BRISQUE, PIQE, Wang–Bovik i wariancja Laplasjanu, które pozostawały odporne na rekodowania i filtry mobilne. Cechy PRNU, CFA oraz podwójna kompresja zwiększały wartość dowodową w materiałach wysokiej jakości. Zbiór cech jakości i artefaktów przetwarzania zachował stabilność w warunkach typowych dla dystrybucji internetowej i może być wykorzystany do kalibracji niepewności oraz walidacji systemów forensycznych.

Wnioski: Zidentyfikowane deskryptory jakości i artefaktów przetwarzania stanowią interpretowalny i odporny fundament detekcji deepfake, łączący cechy percepcyjne i techniczne z fizyką akwizycji. Zbiór danych DFRW umożliwia budowę hybrydowych, wyjaśnialnych detektorów łączących analizę cech IQA z modelami uczenia głębokiego. Przyszłe badania (DFRWv2) skoncentrują się na rozszerzeniu zbioru do ≥ 500 000 klipów z pełnym udziałem modeli dyfuzyjnych i multimodalnością audio-wideo w celu standaryzacji raportowania parametrów θ, p_df, p_real, Δp, PR i 95% CI w analizach forensycznych.

Słowa kluczowe: deepfake, detekcja, jakość obrazu, artefakty przetwarzania, BRISQUE, NIQE, Wang–Bovik, PRNU, podwójna kompresja, DFRW

Typ artykułu: oryginalny artykuł naukowy

Bibliografia:

  1. Brown T., Mann B., Ryder N., Subbiah M., Kaplan J.D., Dhariwal P. i in., Language models are few-shot learners, „Advances in Neural Information Processing Systems” 2020, 33, 1877–1901, https://doi.org/10.48550/arXiv.2005.14165 .
  2. Achiam J., Adler S., Agarwal S., Ahmad L., Akkaya I., Aleman F.L. i in., GPT-4 technical report, 2023, https://doi.org/10.48550/arXiv.2303.08774 .
  3. Perov I., Gao D., Chervoniy N., Li K., Marangond S., Um C. i in., DeepFaceLab: Integrated, flexible and extensible face-swapping framework, 2020, https://doi.org/10.48550/arXiv.2005.05535 .
  4. Roop GitHub, https://github.com/s0md3v/roop [dostęp: 10.10.2025].
  5. Chen R., Chen X., N. B., Ge Y., SimSwap: An efficient framework for high fidelity face swapping, Proceedings of the 28th ACM International Conference on Multimedia, 2020, 2003–2011.
  6. Chesney B., Citron D., Deep fakes: A looming challenge for privacy, democracy, and national security, „California Law Review” 2019, 107, 1753, https://doi.org/10.2139/ssrn.3213954 .
  7. Jędrasiak K., Audio stream analysis for deep fake threat identification, „Civitas et Lex” 2024, 41(1), 21–35, https://doi.org/10.31648/cetl.9684 .
  8. Bikhchandani S., Hirshleifer D., Welch I., A theory of fads, fashion, custom, and cultural change as informational cascades, „Journal of Political Economy” 1992, 100(5), 992–1026, https://doi.org/10.1086/261849 .
  9. Badawy A., Lerman K., Ferrara E., Who falls for online political manipulation?, Companion Proceedings of the 2019 World Wide Web Conference, 2019, 162–168.
  10. DiResta R., The supply of disinformation will soon be infinite, „The Atlantic” 2020, 20(9).
  11. Europol, Online fraud schemes: A web of deceit, IOCTA 2023.
  12. United Nations Office on Drugs and Crime, Emerging threats: The intersection of criminal and technological innovation in the use of automation and AI, Cybercrime Technical Brief Series, 2025.
  13. Federal Bureau of Investigation Internet Crime Complaint Center, Internet Crime Report, FBI IC3, 2023.
  14. Europol, Facing reality? Law enforcement and the challenge of deepfakes, Observatory Report from the Europol Innovation Lab, Publications Office of the European Union, Luxembourg 2022.
  15. Dolhansky B., Bitton J., Pflaum B., Lu J., Howes R., Wang M., Ferrer C.C., The deepfake detection challenge (DFDC) dataset, 2020, https://doi.org/10.48550/arXiv.2006.07397 .
  16. Korshunov P., Marcel S., Deepfakes: A new threat to face recognition? Assessment and detection, 2018, https://doi.org/10.48550/arXiv.1812.08685 .
  17. Wang Z., Bovik A.C., A universal image quality index, „IEEE Signal Processing Letters” 2002, 9(3), 81–84, https://doi.org/10.1109/97.995823 .
  18. Guo C., Pleiss G., Sun Y., Weinberger K.Q., On calibration of modern neural networks, International Conference on Machine Learning, 2017, 1321–1330.
  19. Matern F., Riess C., Stamminger M., Exploiting visual artifacts to expose deepfakes and face manipulations, IEEE Winter Applications of Computer Vision Workshops (WACVW), 2019, 83–92.
  20. Guarnera L., Giudice O., Battiato S., Deepfake detection by analyzing convolutional traces, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020, 666–667.
  21. Farid H., Image forgery detection, „IEEE Signal Processing Magazine” 2009, 26(2), 16–25, https://doi.org/10.1109/MSP.2008.931079 .
  22. Ray S., Applied photographic optics, Routledge, 2002.
  23. Tolosana R., Vera-Rodriguez R., Fierrez J., Morales A., Ortega-Garcia J., Deepfakes and beyond: A survey of face manipulation and fake detection, „Information Fusion” 2020, 64, 131–148, https://doi.org/10.1016/j.inffus.2020.06.014 .
  24. Neves J.C., Tolosana R., Vera-Rodriguez R., Lopes V., Proença H., Fierrez J., GAN fingerprints in face image synthesis, Multimedia Forensics, Springer, Singapore 2022, 175–204.
  25. Agarwal S., Farid H., Gu Y., He M., Nagano K., Li H., Protecting world leaders against deep fakes, CVPR Workshops, 2019.
  26. Verdoliva L., Media forensics and deepfakes: An overview, „IEEE Journal of Selected Topics in Signal Processing” 2020, 14(5), 910–932, https://doi.org/10.1109/JSTSP.2020.3002101 .
  27. Haliassos A., Vougioukas K., Petridis S., Pantic M., Lips don't lie: A generalisable and robust approach to face forgery detection, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, 5039–5049.
  28. Rössler A., Cozzolino D., Verdoliva L., Riess C., Thies J., Nießner M., FaceForensics++: Learning to detect manipulated facial images, Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, 1–11.
  29. Li Y., Yang X., Sun P., Qi H., Lyu S., Celeb-DF: A large-scale challenging dataset for deepfake forensics, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, 3207–3216.
  30. Jiang L., Li R., Wu W., Qian C., Loy C.C., DeeperForensics-1.0: A large-scale dataset for real-world face forgery detection, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, 2889–2898.
  31. Ho J., Jain A., Abbeel P., Denoising diffusion probabilistic models, Proceedings of the 34th International Conference on Neural Information Processing Systems, 2020, 33, 6840–6851.
  32. Mildenhall B., Srinivasan P.P., Tancik M., Barron J.T., Ramamoorthi R., Ng R., NeRF: Representing scenes as neural radiance fields for view synthesis, „Communications of the ACM” 2021, 65(1), 99–106, https://doi.org/10.1145/3503250 .
  33. Durall R., Keuper M., Keuper J., Watch your up-convolution: CNN-based generative deep neural networks are failing to reproduce spectral distributions, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, 7890–7899.
  34. Ricker J., Damm S., Holz T., Fischer A., Towards the detection of diffusion model deepfakes, https://doi.org/10.48550/arXiv.2210.14571 .
  35. Yang X., Li Y., Lyu S., Exposing deep fakes using inconsistent head poses, ICASSP 2019 – IEEE International Conference on Acoustics, Speech and Signal Processing, 2019, 8261–8265.
  36. Wang T., Liao X., Chow K.P., Lin X., Wang Y., Deepfake detection: A comprehensive survey from the reliability perspective, „ACM Computing Surveys” 2024, 57(3), 1–35, https://doi.org/10.1145/3699710 .
  37. Cozzolino D., Pianese A., Nießner M., Verdoliva L., Audio-visual person-of-interest deepfake detection, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, 943–952.
  38. Tariq S., Lee S., Woo S., One detector to rule them all: Towards a general deepfake attack detection framework, Proceedings of the Web Conference, 2021, 3625–3637.
  39. Lu Y., Luo R., Ebrahimi T., A novel framework for assessment of learning-based detectors in realistic conditions with application to deepfake detection, 2022, https://doi.org/10.48550/arXiv.2203.11797 .
  40. Cao X., Gong N.Z., Understanding the security of deepfake detection, International Conference on Digital Forensics and Cyber Crime, Springer, Cham 2021, 360–378.
  41. Ju Y., Jia S., Ke L., Xue H., Nagano K., Lyu S., Fusing global and local features for generalized AI-synthesized image detection, IEEE International Conference on Image Processing, 2022, 3465–3469.
  42. Carlini N., Jagielski M., Choquette-Choo C.A., Paleka D., Pearce W., Anderson H. i in., Poisoning web-scale training datasets is practical, IEEE Symposium on Security and Privacy, 2024, 407–425.
  43. Doshi-Velez F., Kim B., Towards a rigorous science of interpretable machine learning, 2017, https://doi.org/10.48550/arXiv.1702.08608 .
  44. Selvaraju R.R., Cogswell M., Das A., Vedantam R., Parikh D., Batra D., Grad-CAM: Visual explanations from deep networks via gradient-based localization, Proceedings of the IEEE International Conference on Computer Vision, 2017, 618–626.
  45. Odena A., Dumoulin V., Olah C., Deconvolution and checkerboard artifacts, „Distill” 2016, 1(10), e3, https://doi.org/10.23915/distill.00003 .
  46. Zhang K., Zuo W., Zhang L., Learning a single convolutional super-resolution network for multiple degradations, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, 3262–3271.
  47. Mittal A., Moorthy A.K., Bovik A.C., No-reference image quality assessment in the spatial domain, „IEEE Transactions on Image Processing” 2012, 21(12), 4695–4708, https://doi.org/10.1109/TIP.2012.2214050 .
  48. Teed Z., Deng J., RAFT: Recurrent all-pairs field transforms for optical flow, European Conference on Computer Vision, Springer, Cham 2020, 402–419.
  49. Chung J.S., Zisserman A., Lip reading in the wild, Asian Conference on Computer Vision, Springer, Cham 2016, 87–103.
  50. Prajwal K.R., Mukhopadhyay R., Namboodiri V.P., Jawahar C.V., A lip sync expert is all you need for speech to lip generation in the wild, Proceedings of the 28th ACM International Conference on Multimedia, 2020, 484–492.
  51. Baltrušaitis T., Ahuja C., Morency L.P., Multimodal machine learning: A survey and taxonomy, „IEEE Transactions on Pattern Analysis and Machine Intelligence” 2019, 41(2), 423–443, https://doi.org/10.1109/TPAMI.2018.2798607 .
  52. Frank J., Eisenhofer T., Schönherr L., Fischer A., Kolossa D., Holz T., Leveraging frequency analysis for deep fake image recognition, International Conference on Machine Learning, 2020, 3247–3258.
  53. Mittal A., Soundararajan R., Bovik A.C., Making a “completely blind” image quality analyzer, „IEEE Signal Processing Letters” 2012, 20(3), 209–212, https://doi.org/10.1109/LSP.2012.2227726 .
  54. Venkatanath N., Praneeth D., Sumohana S.C., Swarup S.M., Blind image quality evaluation using perception-based features, IEEE 21st National Conference on Communications (NCC), 2015, 1–6.
  55. Saad M.A., Bovik A.C., Charrier C., Blind image quality assessment: A natural scene statistics approach in the DCT domain, „IEEE Transactions on Image Processing” 2012, 21(8), 3339–3352, https://doi.org/10.1109/TIP.2012.2191563 .
  56. Narvekar N.D., Karam L.J., A no-reference image blur metric based on the cumulative probability of blur detection (CPBD), „IEEE Transactions on Image Processing” 2011, 20(9), 2678–2683, https://doi.org/10.1109/TIP.2011.2131660 .
  57. Pech-Pacheco J.L., Cristóbal G., Chamorro-Martinez J., Fernández-Valdivia J., Diatom autofocusing in brightfield microscopy: A comparative study, IEEE Proceedings of the 15th International Conference on Pattern Recognition (ICPR), 2000, Vol. 3, 314–317.
  58. Pertuz S., Puig D., Garcia M.A., Analysis of focus measure operators for shape-from-focus, „Pattern Recognition” 2013, 46(5), 1415–1432, https://doi.org/10.1016/j.patcog.2012.11.011 .
  59. ISO 12233:2017. Photography — Electronic still picture imaging — Resolution and spatial frequency responses.
  60. ITU-T H.264:2019. Advanced Video Coding for Generic Audiovisual Services.
  61. ISO/IEC 23008-2:2017. Information technology — High efficiency video coding.
  62. Damera-Venkata N., Kite T.D., Geisler W.S., Evans B.L., Bovik A.C., Image quality assessment based on a degradation model, „IEEE Transactions on Image Processing” 2000, 9(4), 636–650, https://doi.org/10.1109/83.841940 .
  63. Mahdian B., Saic S., Detection of copy–move forgery using a method based on blur moment invariants, „Forensic Science International” 2007, 171(2–3), 180–189, https://doi.org/10.1016/j.forsciint.2006.11.002 .
  64. Bianchi T., Piva A., Image forgery localization via block-grained analysis of JPEG artifacts, „IEEE Transactions on Information Forensics and Security” 2012, 7(3), 1003–1017, https://doi.org/10.1109/TIFS.2012.2187516 .
  65. Amerini I., Ballan L., Caldelli R., Del Bimbo A., Serra G., A SIFT-based forensic method for copy–move attack detection and transformation recovery, „IEEE Transactions on Information Forensics and Security” 2011, 6(3), 1099–1110, https://doi.org/10.1109/TIFS.2011.2129512 .
  66. Rublee E., Rabaud V., Konolige K., Bradski G., ORB: An efficient alternative to SIFT or SURF, IEEE International Conference on Computer Vision, 2011, 2564–2571.
  67. Lukas J., Fridrich J., Goljan M., Digital camera identification from sensor pattern noise, „IEEE Transactions on Information Forensics and Security” 2006, 1(2), 205–214, https://doi.org/10.1109/TIFS.2006.873602.
  68. Popescu A.C., Farid H., Exposing digital forgeries by detecting duplicated image regions, Computer Science Technical Reports, Dartmouth College, 2004.
  69. Benjamini Y., Hochberg Y., Controlling the false discovery rate: A practical and powerful approach to multiple testing, „Journal of the Royal Statistical Society: Series B” 1995, 57(1), 289–300, https://doi.org/10.1111/j.2517-6161.1995.tb02031.x.
  70. EU DisinfoLab Annual Conference. Conference Dossier 2023, https://www.disinfo.eu/wp-content/uploads/2023/11/Disinfo2023-conference-dossier.pdf [dostęp: 10.10.2025].
  71. Lowe D.G., Distinctive image features from scale-invariant keypoints, „International Journal of Computer Vision” 2004, 60(2), 91–110, https://doi.org/10.1023/B:VISI.0000029664.99615.94.