Page 1 of 21
Advances in Social Sciences Research Journal – Vol. 9, No. 2
Publication Date: February 25, 2022
DOI:10.14738/assrj.92.11738. Seguchi, M., Harada, F., & Shimakawa, H. (2022) Figuring Out Commercial Gimmicks Influencing Consumer Engagement from
Psychological Change Points. Advances in Social Sciences Research Journal, 9(2). 190-210.
Services for Science and Education – United Kingdom
Figuring Out Commercial Gimmicks Influencing Consumer
Engagement from Psychological Change Points
Motoki Seguchi
College of Information Science and Engineering
Ritsumeikan University, Kusatsu, Shiga, Japan
Connect Dot Ltd., Kyoto, Japan
Fumiko Harada
College of Information Science and Engineering
Ritsumeikan University, Kusatsu, Shiga, Japan
Connect Dot Ltd., Kyoto, Japan
Hiromitsu Shimakawa
College of Information Science and Engineering
Ritsumeikan University, Kusatsu, Shiga, Japan
Connect Dot Ltd., Kyoto, Japan
ABSTRACT
Although the scale is not so large, there are many companies that carefully
manufacture traditional products of high quality such as crafts and foods. For such
companies, story-telling commercials explaining their elaborate products on their
SNS sites are attractive means to enlarge their business scale, because it takes a
lower cost than TV commercials. However, because of limited budgets, they make
commercials for themselves. It is difficult for them to evaluate whether their
commercials have an impact on consumers. To compose hand-made commercials
appealing to new customers, this study proposes a method to estimate which
gimmicks in a commercial affect consumer engagement. Customer targeting is
important in business for the small amount of production. The study evaluates the
method, dividing target consumers into two groups, students and working adults.
The change point detection is applied to pupil size for estimating gimmicks affecting
emotional engagement, so is to EDA for estimating ones affecting cognitive
engagement. The impacts on behavioral engagement are evaluated with a
questionnaire. Experiments show that characters are an important gimmick for
cognitive engagement. A climax should be placed firmly in the story to influence
emotional engagement. Flashback scenes also turn out useful gimmicks for both
cognitive and emotional engagement. Furthermore, scenes that allow consumers to
imagine themselves using the product are influential to behavioral engagement.
Specific elements working well for each gimmick turn out to vary with students and
adults. The method manifests gimmicks to be included in a commercial when
companies compose their commercials for themselves.
Keywords: Change point detection; Estimating gimmicks; Consumer engagement;
Psychological changes; Watching commercials; Physiological characteristics.
Page 2 of 21
191
Seguchi, M., Harada, F., & Shimakawa, H. (2022) Figuring Out Commercial Gimmicks Influencing Consumer Engagement from Psychological Change
Points. Advances in Social Sciences Research Journal, 9(2). 190-210.
URL: http://dx.doi.org/10.14738/assrj.92.11738
INTRODUCTION
There are many companies that elaborate on traditional products of high quality such as crafts
and foods, though their business scale is not so large. Many of these companies have clear
concepts for their products. Commercials can be used as a means of appealing the concepts,
which leads to spreading the products. One of the ways to create a commercial is to hire a
commercial production company. However, this way is not practical for those companies
because it requires a large amount of capital. Instead, the companies prefer storytelling
commercials explaining their products on their SNS sites, because it can convey their
elaboration at a far lower cost [1]. As another aspect, to appeal the concepts on their products
effectively, targeting is important, which is a way to narrow down consumers. Companies with
limited financial resources need to create their own storytelling commercials, targeting their
consumers.
Commercials successful in attracting consumers must contain many gimmicks, which make
consumers create good impressions of advertised goods, services, and companies themselves.
For local companies to create their commercials, they need to understand the gimmicks in the
commercial that will affect the target audience.
Consumers would present engagement when they get impacts from commercials. Laurence et
al. [2] divided consumer engagement into three categories: cognitive, affective, and behavioral.
They studied what type of commercials affects a specific type of consumer engagement. The
work is useful to understand the type of consumer engagement individual commercials affect.
However, to evaluate the commercials, the work used only comments consumers left. All of the
comments are written down after the consumers finish watching the entire commercial. While
the work by Laurence is useful for evaluating the entire commercial, it cannot tell us which
gimmicks within the commercial influence the consumers. Furthermore, the study does not
consider target consumers the commercials must give strong impacts.
Commercial can affect consumer engagement. Therefore, contemporary research on consumer
engagement is expected to significantly advance the study of consumer-brand relationships [3].
However, no studies have focused on the direct impact of complex brand content such as
commercials on consumer engagement [4]. Furthermore, studies of consumer engagement in
connection to storytelling content still lack academic attention [5]. Companies that stick to
elaborated products want to compose their own story-telling commercials on their SNS site.
However, they are amateurs when it comes to creating commercials. They do not know what
gimmicks should be included in the commercial.
This study researches what gimmicks in a commercial affect what kind of consumers for local
companies to create effective commercials on their own. This study proposes a method for
extracting change points of psychological signals from the consumers watching commercials to
estimate commercial gimmicks that affect target consumer engagement. Experiments manifest
various useful gimmicks to influence each of cognitive, emotional, and behavioral engagement.
Elements working well for each gimmick turn out to vary with target consumers. Referring to
the results, even companies with limited budgets can incorporate gimmicks into their
storytelling commercials to appeal to consumers who would hold strong interests in their
elaborated products.
Page 3 of 21
192
Advances in Social Sciences Research Journal (ASSRJ) Vol. 9, Issue 2, February-2022
Services for Science and Education – United Kingdom
ENGAGEMENT AND ITS DETECTION
Three Types of Engagement in Product Browsing
The engagement has been examined in a variety of disciplines, including sociology, psychology,
political science, and organizational behavior [6, 7]. The concept of engagement has also
emerged in the field of marketing, where preliminary studies have shown that highly engaged
consumers may exhibit greater loyalty to a focal brand [8].
Consumer engagement in the field of marketing refers to the engagement of consumers in
browsing for products. Consumer engagement in marketing is defined in the work [9] as “a
psychological state that occurs by virtue of interactive, co-creative customer experiences with
a focal agent/object (e.g., a brand) in focal service relationships”. Consumer engagement
consists of three components: cognitive engagement, emotional engagement, and behavioral
engagement [8, 10]. Cognitive engagement refers to the extent to which consumers are paying
attention to and focusing on an object. Emotional engagement refers to the degree to which a
consumer likes or dislikes an object. For example, if consumers feel that they like a product or
are excited when they see a product, this will affect their emotional engagement. Behavioral
engagement refers to how a consumer behaves toward an object. For example, when
consumers feel that they want a product or want to look at more about a product, behavioral
engagement will be affected.
When consumers browse products, their cognitive engagement is first influenced, then their
emotional engagement, and finally their behavioral engagement [2, 11]. In other words,
consumers purchase products by going through the process of cognition to emotion and
emotion to action. Therefore, it is important to affect consumers' cognitive engagement and
emotional engagement to make them take actions such as purchasing a product.
Evaluating Engagement
When encouraging consumers to purchase a product, it is important to take their engagement
into account. For example, if a consumer is not interested in a product at all, it is unlikely that
the product will be purchased even if the consumer is encouraged to purchase it.
Engagement can be evaluated in two methods. The first is a method of evaluation based on
perception. In this method, engagement is assessed based on human perceptions expressed
through linguistic means. Human perception is self-reported by users through interviews,
questionnaires, and dialing devices [12]. This method has been discussed as a psychometric
scale for a long time [13]. However, this method of measuring engagement not only relies on
spontaneous human responses but may also interfere with natural human watching of
commercials. While this method is useful for evaluating behavioral engagement, such as "I want
the product," it is difficult to measure the timing of human interest or the moment of emotional
impact.
The second method is to evaluate based on physiological characteristics. Physiological
characteristics derive from the structural information of the human body. Examples of
physiological characteristics include pupil expansion and contraction, heart rate, and Electro
Dermal Activity (EDA). It is difficult for humans to change physiological characteristics
intentionally. To accurately evaluate human cognitive and emotional engagement, it is more
useful to use physiological characteristics rather than perception-based assessment methods.
Page 4 of 21
193
Seguchi, M., Harada, F., & Shimakawa, H. (2022) Figuring Out Commercial Gimmicks Influencing Consumer Engagement from Psychological Change
Points. Advances in Social Sciences Research Journal, 9(2). 190-210.
URL: http://dx.doi.org/10.14738/assrj.92.11738
Change Finder
In time-series data, change point detection is applied to estimate the time point when some
parameter has changed. Its application to physiological characteristics acquired over time
contributes to the investigation of consumer engagement. Change Finder is a method for
detecting change points by constructing AR models using partial time series models. It
calculates change point scores based on the difference of observed values from predicted ones
[14].
Suppose an input is a multidimensional vector. The �-dimensional AR model is represented by
the following equation (1).
�! = � + &∅"�!#"
$
"%&
+∈!, ∈! ~�. �. (∑) (1)
Where � is the observed value, � is the intercept, ∅ is the autoregressive coefficient, and ∈ is
the white noise. Here, ∅" ∈ �'×'(� = 1, ⋯ , �) is a parameter matrix of dimension � × � and
�!#$:!#& = 9�!#$, ⋯ , �!#&:′��'×$. The probability density function of �!, represented by the AR
model, is expressed by the following equation (2).
�9�!=�!#$:!#&; �: = 1
(2�)'⁄*|∑|&⁄* exp F− 1
2 (�! − �),
∑#&(�! − �)I (2)
Here, � = ∑ ∅"
$
"%& �!#" + �, and the parameter set for the model is � = J∅&, ⋯ , ∅$, �, ∑K. When
the score is calculated by taking the difference between the calculated partial AR series and the
time series data, it is sensitive to noise and has a high probability of producing false positives.
Change Finder applies the AR model in two steps to make the change point score estimation
robust through smoothing. The details of the two-phase estimation method are as follows:
• Learning step 1
1. Learning with AR models:
Prepare and train an AR model as a probability model for time series data. Let �!(�)(� =
1, ⋯ , �) be the probability density function sequence obtained after learning. Here, �!#&(�) is
the probability density function learned from �&:!#& = {�&, ⋯ , �!#&}.
2. Calculation of the logarithmic loss of the outlier score:
Calculate the outlier score of the data �! at each time point � with the logarithmic loss shown in
equation (3).
�����(�!) = − log �!#&(�!) (3)
3. Smoothing:
Letting � be a given positive integer, the outlier scores calculated in 2 are averaged for the data
in the time window of width �. This is called smoothing. This operation is carried out while
shifting the window to construct a moving average series of scores. For the score series
�����(�!)(� = � − � + 1, ⋯ ,�), �-average score series �! is defined in equation (4).
�! = 1
� & �����(�")
!
"%!#-.&
(4)
Page 5 of 21
194
Advances in Social Sciences Research Journal (ASSRJ) Vol. 9, Issue 2, February-2022
Services for Science and Education – United Kingdom
• Learning step 2
1. Learning with AR model:
Apply the AR model to the newly constructed time-series data �!(� = 1, ⋯ , �) in the learning
step 1. Let �!(� = 1, ⋯ , �) be the time series of the stochastic model obtained by this operation.
2. Calculation of �,
-average score:
Let �′ be a given positive integer and find the �′-average score at time � by the following
logarithmic loss.
�����(�) = 1
�, & (− log �"#&(�"))
!
"%!#-!.&
(5)
The higher the value of �����(�) in equation (5), the higher the degree to which time � is a
change point. Change Finder can remove noise-responsive outliers by smoothing outlier scores
calculated in learning step 1 to detect only intrinsic variability in learning step 2. When � is
small in Equation (4), the change point can be detected immediately after it appears, but it is
sensitive to temporary fluctuations in numerical values. On the other hand, when � is large, the
time delay until the change point is detected becomes large, but outliers are filtered out so that
only the change point can be detected.
The Three Elements of Storytelling Content
To promote products to consumers, it is possible to provide commercials with storytelling.
Storytelling content always contains a story plot, characters, and verisimilitude [15, 16, 17].
A story plot is the temporal sequence of events that affect a character. It results from the
interaction of two components of narrative structure: chronology and causality [2]. These
structural elements provide a setting with social, physical, and temporal elements, while the
story is further brought out by the listener through the use of mental imagery, whereby the
recipient of the story becomes part of the story to live it from the inside [18]. In this way, the
meaning of the events in a story derives from the plot of the story, which enables the listener to
mentally construct the plot of the story [16].
The character, the protagonist of the story, is how the listener of the story experiences beliefs,
purposes, and emotions [19]. Characters play a fundamental role in creating empathy between
the listener and the events of the story. Together with the story plot, they help to make the story
resemble real-life experiences [5].
Verisimilitude indicates the likelihood that the events in the story will happen. The perception
of the authenticity of a story is an important factor for the listener to trust and accept the story
and thus engage in the plot. The degree of verisimilitude affects the perception that a story
resembles real-life experiences and increases the listener's narrative transport [20].
Commercial Evaluation by Psychological Change Points
This study proposes a method to reveal gimmicks of commercials that have succeeded in
bringing about consumer engagement. Suppose a small company, which is unfamiliar with
commercial making, challenges it. The company wants to know whether their hand-made
commercials have expected impacts on target consumers. This method enables the company to
evaluate it.
Page 6 of 21
195
Seguchi, M., Harada, F., & Shimakawa, H. (2022) Figuring Out Commercial Gimmicks Influencing Consumer Engagement from Psychological Change
Points. Advances in Social Sciences Research Journal, 9(2). 190-210.
URL: http://dx.doi.org/10.14738/assrj.92.11738
The proposed method allows companies to evaluate their own commercials under the following
scenarios. After determining its target consumers, the company invites some of them to the lab.
It lets them wear sensors to find out which of the gimmicks of the commercials they have
prepared succeeds in influencing them. The consumer watches the prepared commercials on a
PC. When the consumer is watching the commercials, it collects the pupil size and EDA as the
consumer's physiological characteristics. When a commercial attracts or impresses consumers,
their physiological characteristics would change. The method applies Change Finder to the
transition of the pupil size. It examines which gimmicks in the commercial affect the consumer's
cognitive engagement. Change Finder is also applied to the transition of the EDA to uncover
gimmicks that bring the consumers emotional engagement. To examine an impact on
behavioral engagement, it uses questionnaires that consumers complete after watching the
commercials. The overall diagram of the proposed method is shown in Figure 1.
Evaluation of Gimmicks in Commercials
Commercials include gimmicks that may affect consumer engagement. Gimmicks that influence
consumers include what gets attention from consumers, what encourages empathy, and what
provides simulated experiences. For example, a gimmick that captures the attention of
consumers will affect their cognitive engagement, while a gimmick that encourages empathy in
consumers will affect their emotional engagement. This method uses the consumers’
physiological characteristics and questionnaires to estimate the gimmicks in the commercial
that affect consumer engagement.
Figure 1. Method Outline
The gimmicks that affect consumers' cognitive engagement are detected through change-points
of the size of consumers' pupils. The sympathetic nervous system of the autonomic nervous