Runway to Realway: Visual Analysis of Fashion

GeoStyle: Discovering Fashion Trends and Events

Utkarsh Mall

Abstract

Understanding fashion styles and trends is of bully potential interest to retailers and consumers alike. The photos people upload to social media are a historical and public data source of how people dress across the earth and at different times. While we now have tools to automatically recognize the habiliment and style attributes of what people are wearing in these photographs, we lack the ability to analyze spatial and temporal trends in these attributes or brand predictions about the future. In this paper we address this need by providing an automatic framework that analyzes large corpora of street imagery to (a) detect and forecast long-term trends of various fashion attributes as well as automatically discovered styles, and (b) identify spatio-temporally localized events that touch on what people wear. We show that our framework makes long term trend forecasts that are more accurate than prior art, and identifies hundreds of socially meaningful events that impact fashion across the globe. The supplementary material can be found at https://geostyle.cs.cornell.edu/static/pdf/supplementary.pdf

one Introduction

Each day, we collectively upload to social media platforms billions of photographs that capture a broad range of human life and activities around the world. At the same time, object detection, semantic segmentation, and visual search are seeing rapid advances[thirteen] and are being deployed at calibration[22]. With large-scale recognition bachelor as a primal tool in our vision toolbox, information technology is now possible to ask questions near how people dress, eat, and group beyond the world and over time. In this newspaper we focus on how people dress. In item, we inquire: tin can we detect and predict manner trends and styles over space and time?

Nosotros reply these questions by designing an automated method to characterize and predict seasonal and yr-over-year way trends, detect social events (east.g., festivals or sporting events) that impact how people dress, and place social-effect-specific style elements that epitomize these events. Our approach uses existing recognition algorithms to place a coarse set of fashion attributes in a large corpus of images. We then fit interpretable parametric models of long-term temporal trends to these fashion attributes. These models capture both seasonal cycles as well as changes in popularity over time. These models non only help in understanding existing trends, but can likewise make up to 20% more accurate, temporally fine-grained forecasts across long time scales compared to prior methods[1]. For case, we find that year-on-year more people are wearing black, but that they tend to exercise so more in the winter than in the summer.

Our framework not only models long-term trends, just too identifies sudden, curt-term changes in popularity that buck these trends. Nosotros discover that these outliers ofttimes stand for to festivals, sporting events, or other large social gatherings. We provide a methodology to automatically discover the events underlying such outliers past looking at associated image tags and captions, thus tying visual analysis to text-based discovery. We find that our framework finds understandable reasons for all of the near salient events information technology discovers, and in and then doing surfaces intriguing social events around the world that were unknown to the authors. For instance, it discovers an unusual increase in the colour xanthous in Bangkok in early on December, and associates it with the words "begetter", "day", "king", "live", and "dad". This corresponds to the male monarch's altogether, celebrated as Father's Twenty-four hour period in Thailand by wearing yellow[36]. Our framework similarly surfaces events in Ukraine (Vyshyvanka Day), Indonesia (Batik Twenty-four hour period), and Japan (Aureate Week). Figure 1 shows more of the worldwide events discovered by our framework and the clothes that people wear during those events.

Figure one: Major events discovered by our framework. For each consequence, the figure shows the clothing that people typically article of clothing for that event, along with the city, 1 of the months of occurrence, and the virtually descriptive word extracted using the images captions. The inset image shows more precise locations of these cities.

We further show that nosotros can predict trends and events not merely at the level of individual fashion attributes (such as "wearing yellow"), but too at the level of styles consisting of recurring visual ensembles. These styles are identified past clustering photographs in feature infinite to reveal manner clusters: clusters of people dressed in a similar style. Our forecasts of the futurity popularity of styles are just every bit accurate equally our predictions of individual attributes. Further, nosotros tin can run the aforementioned issue detection framework described higher up on style trends, allowing u.s. to not only automatically discover social events, but also associate each event with its own distinctive style; a stylistic signature for each event.

Our contributions, highlighted in Figure two, include:

We present an automated framework for analyzing the temporal behavior of manner elements across the globe. Our framework models and forecasts long-term trends and seasonal behaviors. It also automatically identifies brusque-term spikes caused by events similar festivals and sporting events.
Our framework automatically discovers the reasons behind these events by leveraging textual descriptions and captions.
We connect events with signature styles past performing this analysis on automatically discovered manner clusters.

two Related work

Visual understanding of article of clothing. There has been all-encompassing contempo work in computer vision on characterizing clothing. Some of this work recognizes attributes of people'due south clothing, such as whether a shirt has short or long sleeves[6, v, 4, 42, 19, 23]. Other work goes beyond coarse image-level labels and attempts to segment different clothing items in images[39, 38, twoscore]. Product identification is an "instance-level classification" task used for detecting specific clothing products in photos[7, 33, 12]. Finally, at that place is as well prior piece of work on classifying the "style": the ensemble of wear a person is wearing, east.yard., "hipster", "goth" etc.[18]. In some cases, these labels might be unknown and require discovery[23, fifteen], often by leveraging embeddings of images learnt past attribute recognition systems.

Our piece of work borrows from the attribute and style literature. We make use of several human-annotated attributes on a pocket-size dataset to class an embedding space for the exploration of a much larger set up of images. We use the embedding infinite to characterization attributes and styles over a vast internet-scale dataset. However, our goal is non the labeling itself, but the discovery of interesting geo-temporal trends and their associated styles.

Visual discovery. Although less common, there has been some prior research into using visual analysis to place trends. Early piece of work used low-level image features or mined visually distinctive patches[9, 29, 8] to predict geo-spatial backdrop such every bit perceived condom of cities[ii, 25, 26], or ecological properties such as snowfall or cloud cover[41, 34, 24]. Advances in visual recognition has enabled more than sophisticated analysis, such as the assay of demographics by recognizing the make and model of cars in Street View[10]. However, while this work is exciting, the focus has been on using vision to predict known geo-spatial trends rather than discover new ones. The notion of using visual recognition to ability discovery and prediction of the future is under-explored. Some initial enquiry in this regard has focused on faces[16, 27, 11] and on human activities in a healthcare setting[21]. However, this prior work has generally focused on descriptive analytics and transmission exploration of the data to detect interesting trends. By contrast, we suggest an automated, quantitative framework for both long-term forecasting and discovery. While our work focuses on the fashion domain, our ideas might exist adapted to other applications every bit well.

Trend assay in fashion. Trend analysis has also been practical to the fashion domain, the focus of our work. Frequently, prior work has considered pocket-size datasets such as catwalk images from NYC fashion shows[xiv]. Where larger datasets have been analyzed, interesting trends have been discovered, such equally a sudden increment in popularity for "heels" in Manila[28] or seasonal trends related to styles such as "floral", "pastel", and "neon"[33]. Matzenet al.[23] significantly expand the scope of such trend discovery by leveraging publicly available images uploaded to social media. Nosotros build upon the StreetStyle dataset in this piece of work. However, the assay of the spatial and temporal trends in these papers is often descriptive, and their employ for discovery requires significant transmission exploration. The get-go problem is partly addressed past Al-Halah et al.[1], who attempt to make quantitative forecasts of fashion trends, but whose temporal models are express in their expressivity, forcing them to make very coarse yearly predictions for merely one yr in advance. In contrast, we propose an expressive parametric model for trends that makes much higher quality, fine-grained weekly predictions for as much as 6 months in advance. In add-on, nosotros suggest a framework that automates discovery past automatically surfacing interesting outlier events for analysis.

3 Method

Figure ii: Approach overview. (a) Attribute recognition and style discovery[23] on internet images from multiple cities gives u.s. temporal trends. (b) We fit interpretable parametric models to these trends to characterize and *forecast* (red bend is the fitted trend used to forecast). (c) Deviations from parametric models are identified as events (red points). (d) We place text and styles specific to each event.

Our overall pipeline is shown in Figure2. We first describe our dataset and fashion attribute recognition pipeline, which we suit from StreetStyle[23] so depict our trend analysis and event detection pipeline.

3.1 Background: dataset and attribute recognition

Our dataset uses photos from two social media websites, Instagram and Flickr. In particular, we start with the Instagram-based StreetStyle dataset of Matzenet al.[23] and extend information technology to include photos from the Flickr 100M dataset[32]. The aforementioned pre-processing applied to StreetStyle is also practical to Flickr 100M, including categorization of photos into 44 major world cities across vi continents, person body and face detection, and canonical cropping. Please refer to[23] for details. In full, our dataset includes 7.7 million images of people from around the earth.

Matzenet al. also collect clothing attribute annotations on a 27k subset of the StreetStyle dataset[23]. As in their work, nosotros use these annotations to railroad train a multi-task CNN (GoogLeNet[31]) where separate heads predict separate attributes, e.chiliad., 1 caput may predict "long-sleeves" whereas another may predict "mostly yellow". This training besides has the consequence of automatically producing an embedding of images in the penultimate layer of the network that places similar habiliment attributes and combinations of these attributes, henceforth refered to as "styles", into the same region of the embedding vector infinite.

We take these aspect classifiers and apply them to the total unlabeled gear up of 7.7M of people images. We produce a temporal tendency for each attribute in each city past calculating, for each week, the mean probability of an attribute across all photos from that week and city. Per-image probabilities are derived from the CNN prediction scores after calibration via isotonic regression on a validation set[23].

iii.2 Characterizing trends

Effigy 3: Ii examples of observed trends. As can be seen, trends ofttimes have seasonal variations, but periodic trends are not necessarily sinusoidal. Trends can too involve a linear component (e.grand., the decrease in the incidence of Dresses in Cairo over time). The green confined point the 95% confidence interval for each calendar week.

Given each weekly clothing aspect trend in each city, we wish to (a) narrate this trend in a homo-interpretable manner, and (b) make accurate forecasts about where the trend is headed in the future.

Effigy3 shows ii examples of attribute trends over fourth dimension. We observe several behaviors in these examples. First, there are both fibroid-level trends extending over months or years (e.chiliad., the seasonal cycles in the wearing of multiple layers in Delhi) too every bit fine-scale spikes that occur over days or weeks (east.g., the spike in Dec 2014). 2d, the fibroid tendency oftentimes has a strong periodic component usually governed by different seasons. Third, instead of even sinusoidal upswings and downswings, the periodic trend often consists of up (Figure 3 superlative) or down (Figure three bottom) surges in popularity. Fourth, in some cases this periodic trend is superimposed on a more gradual increase or subtract in popularity, as in Effigy iii (lesser).

Figure 4: Nosotros use a function of the grade every bit our cyclical component because of its ability to model seasonal spikes. This plot shows this function for iii values of and . For ease of comparison, all three functions accept been centered and rescaled to the same dynamic range.

We seek to place both the coarse, slow-irresolute trends that are governed by seasonal cycles or tiresome changes in popularity, as well equally the fine-grained spikes that might arise from events such equally festivals (Christmas, Chinese New year's day) or sporting events (FIFA World Cup). The quondam might tell united states how people in a item place dress in different seasons, while the latter might reveal of import social events with many participants. Nosotros showtime fit a parametric model to capture the slow-changing trends (this section), and and so identify potential events as large departures from the predicted trends (Department3.3).

We model slow-irresolute trends using a parametric model , which is a convex combination of two components: a linear component and a cyclical component:

where the parameter defines the contribution of each component. The linear component, is characterized past slope and intercept :

A standard choice for the cyclical component would exist a sinusoid. However, we want to capture upward and downward surges, then nosotros instead use a more expressive cyclical component of the form:

When is close to , this part behaves like a (shifted) sinusoid, but for higher values of , it has more peaky cycles (Figurefour). and denote period and phase respectively.

The full set of parameters in this parametric model is . Table1 provides intuitive descriptions of these parameters. Considering each parameter is interpretable, our model allows us to non just brand predictions about the future but besides to discover interesting trends and analyze them, as nosotros show in Sectioniv.one.

Tabular array 1: Intuitive descriptions of all parameters

We fit the parameters of the higher up model to the weekly trend of each attribute for each urban center past solving the following not-linear least-squares trouble:

where represents the observed boilerplate probability of the attribute for week in the detail urban center/continent/world and measures the uncertainty of the measurement (reciprocal of the binomial confidence). Nosotros minimize Equation (four) using the Trust Region Reflective algorithm[xx]. To preclude overfitting we ready an upper spring for to continue seasonal variation close to almanac variation. We set up it to , allowing for a maximum of ii complete sinusoidal cycles over a year. We chose 52 because we measure time in weeks.

3.iii Discovering events

Given a fitted model, nosotros now describe how we identify more fine-grained construction in each attribute trend, and correlate these structures with potentially important social gatherings. In particular, we are interested in sharp spikes in popularity of particular kinds of article of clothing, which ofttimes are due to an event. For example, people might habiliment a particular bailiwick of jersey to support their local team on game dark, or wear light-green on St. Patrick's Day.

To discover such events, we start by identifying weeks with big, positive deviations from the fitted model, or outliers, using a binomial hypothesis test. The set of images in calendar week are considered every bit a ready of trials, with those images classified as positive for the aspect constituting "successes" and others failures. The null hypothesis is that the probability of a success is given by the fitted parametric model, . Considering nosotros are interested in positive deviations from this expectation, we use a one-tailed hypothesis exam, where the culling hypothesis is that the truthful probability of success is greater than this expectation. We identify outliers as weeks with -value . We use the reciprocal of the -value, denoted past , equally a measure out of outlier saliency.

We then connect the outliers discovered to the social event that caused them, if whatsoever. To practise so, we note that some of these events might be repeating, annual affairs (such every bit festivals), while others might be ane-off events (e.thou. FIFA World Cup). We therefore formalize an event as a grouping of outliers that are either localized on a few weeks (1-off events) or are separated by a period of approximately a yr (almanac events, like festivals on a solar or a lunisolar agenda[37]).

To decide if our detected outliers fit some event, we need a style to score candidate events. If we take a sequence of outliers for a particular trend in a specific urban center, how do nosotros say if this grouping of outliers is likely to exist an actual event? There are two principal considerations in this decision. First, the outliers involved in the effect must exist salient, that is, they should correspond to significant departures from the background trend. 2nd, they should have the temporal signature described in a higher place: the outliers involved should either exist localized in fourth dimension, or separated by approximately a year.

We formalize this intuition past defining a cost function for each group of outliers such that a smaller cost indicates a higher likelihood of beingness an event. is a product of ii terms: a cost incentivizing the utilise of salient outliers (we use the reciprocal of the boilerplate saliency of the outliers involved), and a cost measuring the difference from the platonic temporal signature:

considers sequent outliers in and assigns a low cost if these consecutive outliers occur very close to each other in time, or are very close to following an almanac cycle. If consecutive events are neither proximal (they are more than weeks apart) nor part of an annual or multi-yr cycle (they miss the bicycle by more than than weeks), the price is set to infinity. Concretely, nosotros define as follows:

Hither, denotes the cardinality of outlier group . is the time difference between sequent outliers, is the length of a year, and measures how far is from an annual cycle. In item,

gives usa a mode of scoring candidate events, simply we yet need to come up upward with a set of candidates in the starting time place from the discovered outliers. There may exist multiple events in a city over time (e.yard., Christmas and Chinese New Year), and nosotros need to separate these events. We consider this as a group problem: given a gear up of outliers occuring at times in the trend of a detail aspect in a detail urban center, nosotros want to partition the gear up into groups. Each group is and then a candidate result. We define the toll of a partition as the average cost of each grouping in the partition, and cull the sectionalisation that minimizes this cost:

This is a combinatorial optimization problem. However, nosotros notice that there are very few outliers for each trend, so this problem tin exist solved optimally using simple enumeration.

Running this optimization trouble for each trend gives the states a gear up of events, each corresponding to a group of outliers. Each upshot is and then associated with a cost . We define the reciprocal of this cost every bit the saliency of the event, and nosotros rank the events in decreasing lodge of their saliency.

Mining underlying causes for events. To derive explanations for each effect, nosotros analyze image captions that accompany the epitome dataset. Nosotros consider images from the relevant location classified as positive for the relevant attribute beyond the year, and split them into ii subsets: those appearing within the outcome weeks, and those at other times. Words actualization in captions of the former only not the latter may indicate why the aspect is more popular specifically in that week. To detect these words, we practise a TF-IDF [30] sorting, considering the captions of the first set equally positive documents and the captions of the 2nd fix as negatives. Images can contribute to a term at most in one case in term frequencies. We perform this analysis using the English language language captions.

iii.4 Style trend analysis

We also wish to identify trends not just in single attributes, but also in combinations of attributes that correspond to looks or styles. Notwithstanding, the number of possible aspect combinations grows exponentially with the number of attributes considered, and near attribute combinations are uninteresting because of their rarity: e.thousand., pinkish, short-sleeved, suits. Instead, we desire to focus on the express set of attribute combinations that are actually prevalent in the data. To do then, nosotros follow the work of Matzenet al.[23] to notice fashion clusters: popular combinations of attributes. Style clusters are identified using a Gaussian mixture model to cluster images in the feature space learned by the CNN. To ensure coverage of all trends while also maintaining sufficient data for each style cluster, we separately notice a small-scale number of style clusters in each urban center. In general, we notice that the style clusters we find correspond to intuitive notions of style. Every bit with individual attributes, our trend assay on these clusters tells u.s.a. non only which styles are coming into or going out of mode, simply also associates styles with major social events (Department 4.3).

iv Results

We now evaluate our ability to discover and predict style events and trends. In add-on, we visualize discovered trends, events, and styles.

4.1 Trend prediction and analysis

Nosotros showtime evaluate our parametric temporal model (Eq.1) based on its ability to make out-of-sample predictions about the future (in-sample predictions are provided in the supplementary). We compare to models proposed by Al-Halahet al.[1], the most relevant prior work. We also compare to 4 ablations of our model: (a) Linear: , (b) Sinusoidal fit: , (c) Cyclic fit: and (d) a linear combination of and . We utilize the same metrics as Al-Halahet al.[ane], namely, MAE and MAPE. The latter looks at the boilerplate absolute error relative to the true trend , expressed as a per centum. However, while Al-Halahet al. only evaluate prediction accuracy in the extreme short term (the very adjacent data point), we consider prediction accuracy both in the short term (side by side data point, or adjacent calendar week) too every bit the long term (next 26 data points, or adjacent half-dozen months). Notation that even though Al-Halahet al. only evaluate predictions over the next information-point, that data betoken corresponds to a full year. Hence they are predicting trends further in the futurity, only their prediction is relatively coarser. We also testify the results of our prediction for more than than i year in supplementary.

We find that our parametric model is significantly amend than all baselines at both long-term and short-term predictions (run into Tabular array 2). Furthermore, the gap between our model and the best method institute by Al-Halah et al. (exponential smoothing) increases when we move to making long-term predictions. We also observe that our model's out-of-sample performance actually matches in-sample operation (shown in supplementary) very well, indicating stiff generalization. This shows that our model generalizes improve and can extrapolate significantly further into the future.

Interestingly, our model is as well significantly improve than the autoregressive baselines. These baselines predict a data point every bit a weighted linear combination of the previous information points, where the weights are learned from data and is cross-validated. Thus, these models have many more parameters than our model (up to 12 more). The fact that our model however performs better suggests that choosing the right parametric form is more than important than simply the size or chapters of the model.

Attribute-based trends

Table two: Comparison of our prediction model against other models from [1]. Mean and Last are naive methods that predict the mean and last of the known time series as the next prediction respectively. AR (autoregression) and VAR (vector-autoregression) are autoregressive methods. ES is exponential smoothing. Lower values are improve.

Interpretability: Our model fitting characterizes each attribute tendency in terms of a few interpretable parameters, shown in Table1, which can exist used in a straightforward fashion to reveal insights. For example, describes the phase of the cyclical trend. If nosotros expect at cities where there is a positive spike in people wearing multiple layers in the winter, and then the peaks should occur in winter months, and cities in the northern and southern hemisphere should be exactly out of phase. Figurefive shows the difference in phase for the multiple-layered habiliment attribute between each pair of cities. We find that cities indeed cluster together based on their hemisphere, with cities in the aforementioned hemisphere closer to each other in phase. Interestingly, cities closer to the equator seem to be half-mode between the two hemispheres and form their own cluster.

As another instance, represents the "spikiness" of the cyclical tendency: a loftier corresponds to a very short-duration increment/decrease in popularity. We can search for aspect trends that testify the spikiest (i.e., highest ) almanac positive spikes. These plough out to be wearing-scarves in Bangkok and article of clothing-category-wearing apparel in Moscow. This might reveal the fact that Bangkok has a very short winter when people wear scarves, while Moscow has a short summer where people wear dresses.

4.2 Event discovery

Later on fitting our parametric tendency model, we discover events using the method discussed in Section 3.three. Our event discovery pipeline detected hundreds of events, detailed in the supplementary. Tabular array3 shows the five most salient events along with the corresponding words associated with the event and a set up of corresponding images. All five stand for to significant social gatherings that some or all of the authors were unaware of a priori :

Male parent's Day in Bangkok is celebrated on the Rex's birthday, and people clothing yellow to honor the king.
FreakNight in Seattle is a trip the light fantastic music event held on or around Halloween. The prevalance of sleeveless clothes is an outlier driven by this event given absurd weather at this time of the year.
Songkran in Bangkok is a festival celebrated in April on the Thai New year and involves people playing with water in warm weather.
The Western Conference Finals of the Stanley Loving cup 2014 in Chicago involved the Chicago Blackhawks and the Los Angeles Kings. People wore their home team's jerseys.
The FIFA World Cup was held in Brazil in 2014 and featured a prevalence of yellow jerseys in support of Brazil.

Note that events such as Father's Twenty-four hour period were farther correctly identified as almanac events.

Effigy five: Phase difference for the *multiple-layered* attribute between 20 cities, using estimated stage parameter .

Table 3: Acme v events detected beyond the earth by finding anomalous behaviour in trends using methods from Section iii.2. The words from the captions of the image posts are sorted by their TF-IDF scores in the associated event week (top-2 are shown). Images from each outcome are sorted based on number of terms in their caption matching the top-5 keywords.

Quantitative evaluation: Quantitative evaluation of our discovered events is challenging because in that location is no dataset or annotations of all the significant social events in the world. However, we can check if the events we discover exercise in fact correspond to real social events, which can be construed as a kind of precision.

To practise this evaluation, we manually inspect each discovered event and the associated tiptop keywords to run into if they reveal an understandable explanation: a real social event. We measure the percentage of events with saliency greater than a threshold for which we found such a reason. Figure 6 shows this pct as a function of the saliency threshold. Nosotros find that 100% of the most salient events and threescore% of all events have explainable reasons, indicating both the power of our model to find events and its ability to identify appropriate keywords for them. Not surprisingly, the per centum of explainable events decreases as event saliency decreases, which validates our model's approximate of saliency every bit a measure of probability of corresponding to a existent-world explainable event.

Figure 6: Left: The percentage of events with saliency greater than a threshold that are explainable, plotted equally the threshold varies. Right: The percentage of events retained when another sample with replacement is used for detection.

Nosotros also evaluate the robustness of our event detector by measuring the stability of detected events across random subsets of data. We resample the dataset 20 times with replacement, and run both the trend characterization and event detection on each subset. We then measure out the fraction of outliers with saliency greater than a threshold in one sampled ready that are notwithstanding salient in a second set. We call this fraction the retentivity, and plot information technology in Effigy 6. Ideally, we want all salient events we detect in one dataset to be detected in all datasets, yielding high retention. Indeed, the high saliency events are retained in other folds. Furthermore, this retention charge per unit increases consistently as the threshold value on saliency increases, indicating that the reciprocal of -value is indeed a practiced measure of the saliency of events.

4.3 Mode tendency assay

Finally, we run the aforementioned trend analysis and event detection pipeline on style clusters. Table 2 shows the prediction error of our parametric tendency analysis compared to various baselines when making long-term fine-grained predictions over the next 26 weeks. We detect that our approach once again significantly outperforms all baselines, and by a larger margin.

Effigy 1 shows the about salient manner-based events for selected cities. We discover that with style clusters, we are able to place events that involve attribute combinations, eastward.thou., people wearing glasses with sleeveless tops during the ACL festival in Austin. More hitting are events such as Durga Puja in Kolkata or Fashion Week in Mumbai which are discovered in spite of the adequately nuanced associated appearance.

four.4 Cross-dataset generalization

We besides show that our method generalizes well to cities not seen during CNN training. We collected Flickr images from Barcelona (a metropolis not in[23]) from 2013 to mid-2018 and fed them through the pipeline described in Section iii. We detected a total of 97k people in these photos.

We examination the predictability of our trend prediction method on this unseen set of images. We used images from 2013 to mid-2017 to fit trends, then predicted the tendency for the final yr of data. Our model (MAE 0.043) performs significantly amend than the best baseline, Autoregression (MAE 0.047), although fitting a sinusoid with a linear component also gives comparable operation (MAE 0.043). We doubtable this is because Barcelona does non see significant variations in weather[35] and hence a smoother sinusoid models the seasonal changes likewise every bit our model.

We also discovered events in Barcelona using the method described in Section 3.3. The pinnacle-nigh event discovered in Barcelona corresponds to people gathering in yellow shirts for the "Catalan Way", a long human chain in support of Catalan independence from Spain, in September 2013 (Figure 7). This upshot is a significant political result, and it validates our framework's ability to identify important social events from raw information beyond multiple datasets and bring them to the fore.

Figure 7: Images from "Catalan Fashion" an event discovered from September 2013 in Barcelona.

v Decision and Future Work

This work has established a framework for automatically analyzing temporal trends in fashion attributes and style past examining millions of photos published publicly to the spider web. We characterized these trends using a new model that is both more interpretable and makes better long-term forecasts. We besides presented a methodology to automatically detect social events that impact how people dress. However, this is but a showtime pace and there are many questions notwithstanding to be answered, such as the identification and mitigation of biases in social media imagery, and the propagation of styles across space. The trouble of analyzing trends is likewise relevant in other visual domains, such as understanding which animals are getting rarer over time in photographic camera trap images[iii] or how country-employ patterns are irresolute in satellite imagery[17]. We therefore believe that this is an important problem deserving of time to come research.

Acknowledgements. This work was funded past NSF (CHS: 1617861 and CHS: 1513967) and an Amazon Research Honor.

References

[1] Ziad Al-Halah, Rainer Stiefelhagen, and Kristen Grauman. Fashion forrad: Forecasting visual style in fashion. In ICCV , 2017.
[2] Sean M Arietta, Alexei A Efros, Ravi Ramamoorthi, and Maneesh Agrawala. Metropolis Forensics: Using visual elements to predict non-visual city attributes. IEEE Trans. Visualization and Computer Graphics , 20(12), Dec 2014.
[iii] Sara Beery, Grant Van Horn, and Pietro Perona. Recognition in terra incognita. In ECCV , 2018.
[4] Lukas Bossard, Matthias Dantone, Christian Leistner, Christian Wengert, Till Quack, and Luc Van Gool. Wearing apparel classification with fashion. In Proc. Asian Conf. on Figurer Vision , 2013.
[5] Lubomir Bourdev, Subhransu Maji, and Jitendra Malik. Describing people: Poselet-based aspect classification. In ICCV , 2011.
[6] Huizhong Chen, Andrew Gallagher, and Bernd Girod. Describing vesture by semantic attributes. In ECCV , 2012.
[7] Wei Di, C. Wah, A. Bhardwaj, R. Piramuthu, and Northward. Sundaresan. Style finder: Fine-grained clothing style detection and retrieval. In Proc. CVPR Workshops , 2013.
[viii] Carl Doersch, Abhinav Gupta, and Alexei A. Efros. Mid-level visual element discovery as discriminative fashion seeking. In NeurIPS , 2013.
[nine] Carl Doersch, Saurabh Singh, Abhinav Gupta, Josef Sivic, and Alexei A. Efros. What makes Paris look similar Paris? SIGGRAPH , 31(4), 2012.
[10] Timnit Gebru, Jonathan Krause, Yilun Wang, Duyun Chen, Jia Deng, Erez Lieberman Aiden, and Li Fei-Fei. Using deep learning and Google Street View to gauge the demographic makeup of neighborhoods across the Us. Proc. National University of Sciences , 2017.
[eleven] Shiry Ginosar, Kate Rakelly, Sarah Sachs, Brian Yin, and Alexei A. Efros. A Century of Portraits: A visual historical tape of american high school yearbooks. In ICCV Workshops , 2015.
[12] Grand Hadi Kiapour, Xufeng Han, Svetlana Lazebnik, Alexander C Berg, and Tamara L Berg. Where to buy it: Matching street article of clothing photos in online shops. In ICCV , 2015.
[13] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In ICCV , 2017.
[xiv] Shintami C. Hidayati, Kai-Lung Hua, Wen-Huang Cheng, and Shih-Wei Sun. What are the fashion trends in New York? In Proc. Int. Conf. on Multimedia , 2014.
[15] Wei-Lin Hsiao and Kristen Grauman. Learning the latent "look": Unsupervised discovery of a style-coherent embedding from fashion images. In ICCV , 2017.
[16] Mohammad T Islam, Connor Greenwell, Richard Souvenir, and Nathan Jacobs. Big-scale geo-facial epitome assay. EURASIP J. on Epitome and Video Processing , 2015(1), 2015.
[17] Neal Jean, Sherrie Wang, Anshul Samar, George Azzari, David Lobell, and Stefano Ermon. Tile2vec: Unsupervised representation learning for spatially distributed information. In AAAI , 2019.
[18] G. Hadi Kiapour, Kota Yamaguchi, Alexander C. Berg, and Tamara Fifty. Berg. Hipster wars: Discovering elements of mode styles. In ECCV , 2014.
[19] Ziwei Liu, Ping Luo, Shi Qiu, Xiaogang Wang, and Xiaoou Tang. Deepfashion: Powering robust clothes recognition and retrieval with rich annotations. In CVPR , 2016.
[20] Ladislav Lukšan. Inexact trust region method for large sparse nonlinear to the lowest degree squares. book 29, 1993.
[21] Zelun Luo, Jun-Ting Hsieh, Niranjan Balachandar, Serena Yeung, Guido Pusiol, Jay Luxenberg, Grace Li, Li-Jia Li, Northward Lance Downing, Arnold Milstein, et al. Reckoner vision-based descriptive analytics of seniors' daily activities for long-term health monitoring. Machine Learning for Healthcare (MLHC) , 2018.
[22] Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens van der Maaten. Exploring the limits of weakly supervised pretraining. arXiv preprint arXiv:1805.00932 , 2018.
[23] Kevin Matzen, Kavita Bala, and Noah Snavely. Streetstyle: Exploring earth-broad clothing styles from millions of photos. CoRR , 2017.
[24] Calvin Murdock, Nathan Jacobs, and Robert Pless. Building dynamic cloud maps from the basis upward. In ICCV , 2015.
[25] Nikhil Naik, Jade Philipoom, Ramesh Raskar, and César Hidalgo. Streetscore: Predicting the perceived safe of one million streetscapes. In Proc. CVPR Workshops , 2014.
[26] Vicente Ordonez and Tamara L. Berg. Learning loftier-level judgments of urban perception. In ECCV , 2014.
[27] Tawfiq Salem, Scott Workman, Menghua Zhai, and Nathan Jacobs. Analyzing homo advent as a cue for dating images. In WACV , 2016.
[28] Edgar Simo-Serra, Sanja Fidler, Francesc Moreno-Noguer, and Raquel Urtasun. Neuroaesthetics in style: Modeling the perception of fashionability. In CVPR , 2015.
[29] Saurabh Singh, Abhinav Gupta, and Alexei A. Efros. Unsupervised discovery of mid-level discriminative patches. In ECCV , 2012.
[xxx] Karen Sparck Jones. A statistical interpretation of term specificity and its application in retrieval. Periodical of documentation , 28(i):eleven–21, 1972.
[31] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In CVPR , 2015.
[32] Bart Thomee, David A. Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. Yfcc100m: The new data in multimedia inquiry. Commun. ACM , 59(two):64–73, Jan. 2016.
[33] Sirion Vittayakorn, Kota Yamaguchi, Alexander C Berg, and Tamara L Berg. Track to Realway: Visual assay of fashion. In WACV , 2015.
[34] Jingya Wang, Mohammed Korayem, and David J. Crandall. Observing the natural earth with flickr. In ICCV Workshops , 2013.
[35] Wikipedia contributors. Climate of Barcelona.
[36] Wikipedia contributors. Father's Day.
[37] Wikipedia contributors. Lunisolar Calendar.
[38] Kota Yamaguchi, Thou Hadi Kiapour, and Tamara L Berg. Paper doll parsing: Retrieving similar styles to parse article of clothing items. In ICCV , 2013.
[39] Kota Yamaguchi, M Hadi Kiapour, Luis E Ortiz, and Tamara L Berg. Parsing clothing in fashion photographs. In CVPR , 2012.
[40] Wei Yang, Ping Luo, and Liang Lin. Habiliment co-parsing by joint image segmentation and labeling. In CVPR , 2014.
[41] Haipeng Zhang, Mohammed Korayem, David J. Crandall, and Gretchen LeBuhn. '. In WWW , 2012.
[42] Ning Zhang, Manohar Paluri, Marc'Aurelio Rantazo, Trevor Darrell, and Lubomir Bourdev. Panda: Pose aligned networks for deep attribute modeling. In CVPR , 2014.