“Face manipulation techniques improve fast with the development of powerful image generation models. two particular face manipulation methods, namely face swap and expression reenactment attract much attention for their flexibility and ease to generate high quality synthesis results. recently, these two subjects are actively studied. however, most existing methods treat the two tasks separately, ignoring their underlying similarity. in this paper, we propose to tackle the two problems within a unified framework that achieves high quality synthesis results. the enabling component for our unified framework is the clean disentanglement of 3d pose, shape, and expression factors and then recombining them for different tasks accordingly. we then use the same set of 2d representations for face swap and expression reenactment tasks that are input to a common image translation model to directly generate the final synthetic images. once trained, the proposed model can accomplish both face swap and expression reenactment tasks for previously unseen subjects. comprehensive experiments and comparisons show that the proposed method achieves high fidelity results in multiple aspects, and it is especially good at faithfully preserving source facial shape in the face swap task, and accurately transferring facial movements in the expression reenactment task.”

“The quality and efficiency of generating face-swap images have been markedly strengthened by deep learning. for instance, the face-swap manipulations by deepfake are so real that it is tricky to distinguish authenticity through automatic or manual detection. to augment the efficiency of distinguishing face-swap images generated by deepfake from real facial ones, a novel counterfeit feature extraction technique was developed based on deep learning and error level analysis (ela). it is related to entropy and information theory such as cross-entropy loss function in the final softmax layer. the deepfake algorithm is only able to generate limited resolutions. therefore, this algorithm results in two different image compression ratios between the fake face area as the foreground and the original area as the background, which would leave distinctive counterfeit traces. through the ela method, we can detect whether there are different image compression ratios. convolution neural network (cnn), one of the representative technologies of deep learning, can extract the counterfeit feature and detect whether images are fake. experiments show that the training efficiency of the cnn model can be significantly improved by the ela method. in addition, the proposed technique can accurately extract the counterfeit feature, and therefore achieves outperformance in simplicity and efficiency compared with direct detection methods. specifically, without loss of accuracy, the amount of computation can be significantly reduced (where the required floating-point computing power is reduced by more than 90%).”

“We consider the problem of face swapping in images, where an input identity is transformed into a target identity while preserving pose, facial expression and lighting. to perform this mapping, we use convolutional neural networks trained to capture the appearance of the target identity from an unstructured collection of his/her photographs. this approach is enabled by framing the face swapping problem in terms of style transfer, where the goal is to render an image in the style of another one. building on recent advances in this area, we devise a new loss function that enables the network to produce highly photorealistic results. by combining neural networks with simple pre- and post-processing steps, we aim at making face swap work in real-time with no input from the user.”

“Face swap is one of the popular machine vision problems recently. important problems in real-time face swap are the need for high computation power and the decrease in success of applications in case of face pose variations. in this paper, an original face swap algorithm which can work in real time is proposed. after detecting two faces in the input image, 68 landmark points of each face are localized. using these points, a facial model has been formed which separates the faces into 23 quadrilateral planar regions. homographies are calculated between the quadrilaterals formed by the same points on both sides in order not to disrupt the holistic structure of the selected regions. face swap is performed by warping the face patches with calculated homographies. with the proposed method, the calculation resources used in the algorithm have been used efficiently and identity information such as eye, mustache and eyebrow has been preserved. the symmetries of the visible regions are used for the invisible parts in the viewing angles that the camera cannot see the entire face. the success of our study has been tested with different camera resolutions and faces at different angles and qualitative results are given.”

“There is a growing concern over the issues related to online privacy due to large availability of high quality images. to tackle the privacy concerns a face swapping application is proposed. there is a library of face images, which is created by downloading images from different sources on internet. for any given image, first of all facial landmarks are detected. the second image is rotated and scaled so that it can properly fit over the input image. to make sure that the new image looks natural, color balance adjustment is done. after that blending of features from the second image onto the input image is done. it is also shown how this system can be used to creating appealing and funny photographs for entertainment purposes. we conclude with a study that shows the high quality of images produced by this system as compared to existing face swap applications and also limitations of this system.”

“Facial landmark points that are precisely extracted from the face images improve the performance of many applications in the domains of computer vision and graphics. face swapping is one of such applications. with the availability of sophisticated image editing tools and the use of deep learning models, it is easy to create swapped face images or face swap attacks in images or videos even for non-professionals. face swapping transfers a face from a source to a destination image, while preserving photo realism. it has potential applications in computer games, privacy protection, etc. however, it could also be used for fraudulent purposes. in this paper, we propose an approach to create face swap attacks and detect them from the original images. the augmented 81-facial landmark points are extracted for creating the face swap attacks. the feature descriptors weighted local magnitude patterns (wlmp) and support vector machines (svm) are utilized for the swapped face images detection. the performance of the proposed approach is demonstrated by different types of svm classifiers on a real-world dataset. experimental results show that the proposed system effectively does face swapping and detection with an accuracy of 95%.”

“Face alignment and segmentation are challenging problems which have been extensively studied in the field of multimedia. these two tasks are closely related and their learning processes are supposed to benefit each other. hence, we present a joint multi-task learning algorithm for both face alignment and segmentation using deep convolutional neural network (cnn). the proposed multi-task learning approach allows cnn model to simultaneously share visual knowledge between different tasks. with a carefully designed refinement residual module, the cross-layer features are fused in a collaborative manner. to the best of our knowledge, this is the first time that face alignment and segmentation are learned together via deep multi-task learning. our experiments show that learning these two related tasks simultaneously builds a synergy between them, improves the performance of each individual task, and rivals recent approaches. furthermore, we demonstrate the effectiveness of our model in two practical applications: virtual makeup and face swap.”

“Abuse of face swap techniques poses serious threats to the integrity and authenticity of digital visual media. more alarmingly, fake images or videos created by deep learning technologies, also known as deepfakes, are more realistic, high-quality, and reveal few tampering traces, which attracts great attention in digital multimedia forensics research. to address those threats imposed by deepfakes, previous work attempted to classify real and fake faces by discriminative visual features, which is subjected to various objective conditions such as the angle or posture of a face. differently, some research devises deep neural networks to discriminate deepfakes at the microscopic-level semantics of images, which achieves promising results. nevertheless, such methods show limited success as encountering unseen deepfakes created with different methods from the training sets. therefore, we propose a novel deepfake detection system, named fakefilter, in which we formulate the challenge of unseen deepfake detection into a problem of cross-distribution data classification, and address the issue with a strategy of domain adaptation. by mapping different distributions of deepfakes into similar features in a certain space, the detection system achieves comparable performance on both seen and unseen deepfakes. further evaluation and comparison results indicate that the challenge has been successfully addressed by fakefilter.”

“Generative deep learning algorithms have progressed to a point where it is difficult to tell the difference between what is real and what is fake. in 2018, it was discovered how easy it is to use this technology for unethical and malicious applications, such as the spread of misinformation, impersonation of political leaders, and the defamation of innocent individuals. since then, these ‘deepfakes’have advanced significantly. in this article, we explore the creation and detection of deepfakes and provide an in-depth view as to how these architectures work. the purpose of this survey is to provide the reader with a deeper understanding of (1) how deepfakes are created and detected, (2) the current trends and advancements in this domain, (3) the shortcomings of the current defense solutions, and (4) the areas that require further research and attention.”

“New developments in artificial intelligence (ai) have significantly improved the quality and efficiency in generating fake face images; for example, the face manipulations by deepfake are so realistic that it is difficult to distinguish their authenticity—either automatically or by humans. in order to enhance the efficiency of distinguishing facial images generated by ai from real facial images, a novel model has been developed based on deep learning and error level analysis (ela) detection, which is related to entropy and information theory, such as cross-entropy loss function in the final softmax layer, normalized mutual information in image preprocessing, and some applications of an encoder based on information theory. due to the limitations of computing resources and production time, the deepfake algorithm can only generate limited resolutions, resulting in two different image compression ratios between the fake face area as the foreground and the original area as the background, which leaves distinctive artifacts. by using the error level analysis detection method, we can detect the presence or absence of different image compression ratios and then use convolution neural network (cnn) to detect whether the image is fake. experiments show that the training efficiency of the cnn model can be significantly improved by using the ela method. and the detection accuracy rate can reach more than 97% based on cnn architecture of this method. compared to the state-of-the-art models, the proposed model has the advantages such as fewer layers, shorter training time, and higher efficiency.”

“Video facial swap usually has strong entertainment applications, and it is also applicable for the post-production of films and has great application value. at present, the popular face swap is done manually by the ps software, and the synthetic effect of the automatic face changing technology is not good. in order to make up for the lack of these features, this paper proposes a method of video face swap based on autoencoder generation network. the network learns the mapping relationship between distorted face and original face: the encoder can distinguish and extract facial information, and the decoder can restore face separately. first, the local information of tow face is sent to the network to get the initial model; then, the global information is put into the network for fine-tuning; finally, the face exchange between a and b is completed with face alignment and alpha fusion. the experimental results show that the quality of the method is improved significantly.”

“Deepfakes are a recent off-the-shelf manipulation technique that allows anyone to swap two identities in a single video. in addition to deepfakes, a variety of gan-based face swapping methods have also been published with accompanying code. to counter this emerging threat, we have constructed an extremely large face swap video dataset to enable the training of detection models, and organized the accompanying deepfake detection challenge (dfdc) kaggle competition. importantly, all recorded subjects agreed to participate in and have their likenesses modified during the construction of the face-swapped dataset. the dfdc dataset is by far the largest currently and publicly available face swap video dataset, with over 100,000 total clips sourced from 3,426 paid actors, produced with several deepfake, gan-based, and non-learned methods. in addition to describing the methods used to construct the dataset, we provide a detailed analysis of the top submissions from the kaggle contest. we show although deepfake detection is extremely difficult and still an unsolved problem, a deepfake detection model trained only on the dfdc can generalize to real ‘in-the-wild’ deepfake videos, and such a model can be a valuable analysis tool when analyzing potentially deepfaked videos. training, validation and testing corpuses can be downloaded from ai.facebook.com/datasets/dfdc.”

“Videos obtained by current face swapping techniques can contain artifacts potentially detectable, yet unobtrusive to human observers. however, the perceptual differences between real and altered videos, as well as properties leading humans to classify a video as manipulated, are still unclear. thus, to support the research on perceived realism and conveyed emotions in face swap videos, this paper introduces a high-resolution dataset providing the community with the necessary sophisticated stimuli. our recording process has been specifically designed to focus on human perception research and entails three scenarios (text-reading, emotion-triggering, and free-speech). we assess the perceived realness of our dataset through a series of experiments. the results indicate that our stimuli are overall convincing, even for long video sequences. furthermore, we partially annotate the dataset with noticeable facial distortions and artifacts reported by participants.”

“The recent advance of synthetic image generation and manipulation methods allows us to generate synthetic face images close to real images. on the other hand, the importance of identifying the synthetic face images increases more and more to protect personal privacy from those. although some deep learning-based image forensic methods have been developed recently, it is still challenging to distinguish synthetic images generated by recent image generation and manipulation methods such as the deep fake, face2face, and face swap. to resolve this challenge, we propose a novel generative adversarial ensemble learning method. we train multiple discriminative and generative networks based on the adversarial learning. compared to the conventional adversarial learning, our method is however more focused on improving the discrimination ability rather than image generation one. to this end, we improve the discriminabilty by ensembling outputs from different two discriminators. in addition, we train two generators in order to generate general and hard synthetic images. by ensemble learning of all the generators and discriminators, we improve the discriminators by using the generated synthetic face images, and improve the generators by passing the combined feedback of the discriminators. on the faceforensics benchmark challenge, we thoroughly evaluate our methods by comparing the recent methods. we also provide the ablation study to prove the effectiveness and usefulness of our method.”

“Deepfake (a bag of ‘deep learning’ and ‘fake’) is a technique for human image synthesis based on artificial intelligence, i.e., to superimpose the existing (source) images or videos onto destination images or videos using neural networks (nns). deepfake enthusiasts have been using nns to produce convincing face swaps. deepfakes are a type of video or image forgery developed to spread misinformation, invade privacy, and mask the truth using advanced technologies such as trained algorithms, deep learning applications, and artificial intelligence. they have become a nuisance to social media users by publishing fake videos created by fusing a celebrity’s face over an explicit video. the impact of deepfakes is alarming, with politicians, senior corporate officers, and world leaders being targeted by nefarious actors. an approach to detect deepfake videos of politicians using temporal sequential frames is proposed. the proposed approach uses the forged video to extract the frames at the first level followed by a deep depth-based convolutional long short-term memory model to identify the fake frames at the second level. also the proposed model is evaluated on our newly collected ground truth dataset of forged videos using source and destination video frames of famous politicians. experimental results demonstrate the effectiveness of our method.”

“Portrait editing is a popular subject in photo manipulation.the generative adversarial network (gan) advances the generating of realistic faces and allows more face editing. in this paper, we argue about three issues in existing techniques: diversity, quality, and controllability for portrait synthesis and editing. to address these issues, we propose a novel end-to-end learning framework that leverages conditional gans guided by provided face masks for generating faces. the framework learns feature embeddings for every face component (e.g., mouth, hair, eye), separately, contributing to better correspondences for image translation, and local face editing. with the mask, our network is available to many applications, like face synthesis driven by mask, face swap+ (including hair in swapping), and local manipulation. it can also boost the performance of face parsing a bit as an option of data augmentation.”

“This paper proposes a cnn-based (convolutional neural network based) network to detect altered face picture, which can cover the most common face swap methods. the network uses an autoencoder which is pre-trained on the original images to reconstruct the input images. the reconstructed one and the input image are then processed by the srm filter which can extract the noise distribution of images. we then feed the minus result of two processed results into a cnn architecture to predict whether the input image is original or tampered. the model was trained and evaluated in faceforensics dataset and state-of-art face swap method. experimental results demonstrate the effectiveness of our network.”

“In recent months a machine learning based free software tool has made it easy to create believable face swaps in videos that leaves few traces of manipulation, in what are known as deepfake videos. scenarios where these realistic fake videos are used to create political distress, blackmail someone or fake terrorism events are easily envisioned. this paper proposes a temporal-aware pipeline to automatically detect deepfake videos. our system uses a convolutional neural network (cnn) to extract frame-level features. these features are then used to train a recurrent neural network (rnn) that learns to classify if a video has been subject to manipulation or not. we evaluate our method against a large set of deepfake videos collected from multiple video websites. we show how our system can achieve competitive results in this task while using a simple architecture.”

“In recent years, with the advancements in the deep learning realm, it has been easy to create and generate synthetically the face swaps from gans and other tools, which are very realistic, leaving few traces which are unclassifiable by human eyes. these are known as ‘Deepfakes’ and most of them are anchored in video formats. such realistic fake videos and images are used to create a ruckus and affect the quality of public discourse on sensitive issues; defaming one’s profile, political distress, blackmailing and many more fake cyber terrorisms are envisioned. this work proposes a microscopic-typo comparison of video frames. this temporal-detection pipeline compares very minute visual traces on the faces of real and fake frames using convolutional neural network (cnn) and stores the abnormal features for training. a total of 512 facial landmarks were extracted and compared. parameters such as eye-blinking lip-synch; eyebrows movement, and position, are few main deciding factors that classify into real or counterfeit visual data. the recurrent neural network (rnn) pipeline learns based on these features-fed inputs and then evaluates the visual data. the model was trained with the network of videos consisting of their real and fake, collected from multiple websites. the proposed algorithm and designed network set a new benchmark for detecting the visual counterfeits and show how this system can achieve competitive results on any fake generated video or image.”

“A new non-iterative mesh generation algorithm named conforming to interface structured adaptive mesh refinement (cisamr) is introduced for creating 3d finite element models of problems with complex geometries. cisamr transforms a structured mesh composed of tetrahedral elements into a conforming mesh with low element aspect ratios. the construction of the mesh begins with the structured adaptive mesh refinement of elements in the vicinity of material interfaces. an r-adaptivity algorithm is then employed to relocate selected nodes of nonconforming elements, followed by face-swapping a small fraction of them to eliminate tetrahedrons with high aspect ratios. the final conforming mesh is constructed by sub-tetrahedralizing remaining nonconforming elements, as well as tetrahedrons with hanging nodes. in addition to studying the convergence and analyzing element-wise errors in meshes generated using cisamr, several example problems are presented to show the ability of this method for modeling 3d problems with intricate morphologies.”