How to analyze the target audience yourself. How to do text analysis – parsing algorithms and examples

Algorithms and heuristics

The described problem of recognition of numbers can be solved by trying to independently select a function that implements the corresponding display. It will work out, most likely, not very quickly and not very well. On the other hand, you can resort to machine learning methods, that is, use a manually labeled sample (or, in other cases, one or another historical data) to automatically select a decision function. Thus, hereinafter, I will call a (generalized) machine learning algorithm an algorithm that, one way or another, based on data, forms a non-deterministic algorithm that solves a particular problem. (The non-determinism of the resulting algorithm is needed so that a reference that uses preloaded data or an external API does not fall under the definition).
Thus, machine learning is the most common and powerful (but, nevertheless, not the only) method of data analysis. Unfortunately, people have not yet invented machine learning algorithms that process data of a more or less arbitrary nature well, and therefore a specialist has to independently pre-process the data to bring it into a form suitable for the application of the algorithm. In most cases, such preprocessing is called feature selection or preprocessing. The fact is that most machine learning algorithms take as input sets of numbers of fixed length (for mathematicians, points in

). However, now a variety of algorithms based on neural networks are also widely used, which can take as input not only sets of numbers, but also objects that have some additional, mainly geometric, properties, such as images (the algorithm takes into account not only pixel values, but also their mutual arrangement), audio, video and texts. Nevertheless, some preprocessing usually occurs in these cases as well, so we can assume that for them feature selection is replaced by a selection of successful preprocessing.
A supervised machine learning algorithm (in the narrow sense of the word) can be called an algorithm (for mathematicians, a mapping) that takes as input a set of points in

(also called examples or samples)

and labels (values ​​we are trying to predict)

, and at the output gives the algorithm (function)

already matching a specific value

any entrance

belonging to the example space. For example, in the case of the above-mentioned neural network that recognizes numbers, using a special procedure based on a training sample, values ​​are set corresponding to the connections between neurons, and with their help, at the stage of application, one or another prediction is calculated for each new example. By the way, the collection of examples and labels is called a training sample.
The list of effective supervised machine learning algorithms (in the narrow sense) is strictly limited and almost never expands despite active research in this area. However, it takes experience and training to apply these algorithms correctly. The issues of effectively reducing a practical task to the task of data analysis, selecting a list of features or preprocessing, a model and its parameters, as well as competent implementation are not easy in themselves, not to mention working on them together.
The general scheme for solving the problem of data analysis using the machine learning method looks like this:

It is convenient to separate the chain “preprocessing – machine learning model – postprocessing” into a single entity. Often, such a chain remains unchanged and only regularly retrains on the newly received data. In some cases, especially in the early stages of project development, its content is replaced by more or less complex heuristics that do not directly depend on data. There are also more tricky cases. We will introduce a separate term for such a chain (and its possible variants) and call it a meta-model. In the case of heuristics, it is reduced to the following scheme: Heuristic is simply a hand-picked function that does not use advanced methods, and, as a rule, does not give a good result, but is acceptable in certain cases, for example, in the early stages of project development.

Supervised machine learning tasks

Depending on the setting, machine learning problems are divided into classification, regression and logistic regression problems.
Classification is a problem statement in which it is required to determine to which class from a certain clearly defined list the incoming object belongs. A typical and popular example is the aforementioned digit recognition, in which each image must be associated with one of 10 classes corresponding to the illustrated digit.
Regression is a problem statement in which it is required to predict some quantitative characteristic of an object, for example, price or age.
Logistic regression combines the properties of the above two problem statements. It specifies the events that have taken place on objects, and it is required to predict their probabilities on new objects. A typical example of such a problem is the problem of predicting the likelihood of a user clicking a referral link or ad.

Typical project development cycle

In the most general terms, the development cycle of a data analysis project is as follows.

1. Study of the problem statement, possible data sources.
2. Reformulation in mathematical language, selection of prediction quality metrics.
3. Writing a pipeline for training and (at least test) use in a real environment.
4. Writing a problem-solving heuristic or simple machine learning algorithm.
5. If necessary, improve the quality of the algorithm, it is possible to refine the metrics, attract additional data.

Comprehensive text analysis

This task includes many parameters by which it is easier to understand a passage from the work. But there is no clear instruction or scheme, although it is necessary to adhere to some kind of plan in order to draw up the text of the analysis, where the conclusion will follow from certain facts, supported by the given arguments.

It is worth starting with the fact that after reading, you need to title the text. So for yourself, you can determine the theme and theme and already at the beginning answer the question: “What did the author want to say with this passage?”

It is worth remembering that the topic is the subject of discussion. And a topic is a collection of topics that can be in the proposed passage.

To help in the analysis, communication means can be used, divided into lexical and morphological. Those. it is necessary to determine whether synonyms, repetitions, conjunctions, verbs and participles are used.

It is also necessary to mention the style of the text, which can be artistic, official-business, scientific or colloquial. And also it should be clarified what type of speech is used: narration, reasoning or description.

Knowing all the points will undoubtedly help in parsing, and the student will no longer ask the question: how to analyze the text. He immediately, according to a certain plan, will begin to investigate the proposed work, and at the end he can easily draw a conclusion with the given arguments.

In Russian language and literature

And finally. Analyzes of texts in the Russian language and literature may differ slightly from each other. If it is taken from any work, you must use several of the same steps. In order:

1. Genre of text – legend, poem, parable, recollection, sketch
2. Text theme – any work has its own theme
3. What methods of constructing the text were used – repetitions, oppositions, amplification, dynamism, contemplation
4. Use of visual media
5. The general impression of what you read – if you read the text thoughtfully, then a certain impression will definitely remain, and it should be told about it at the very end of the analysis

How to analyze a verse?

In order to analyze the poem in a volumetric and accurate manner, it is necessary to draw up a plan in which the features of the work will be analyzed point by point. A rough outline of a poem might be as follows:

• The history of the creation of a poetic text, a description of the events that led to its creation. In this part, you can give a little information about the life of the author. You should also indicate how significant this work is for the poet.
• It is necessary to determine the genre, subject and theme of the work. The subject of the image in a lyric work is the inner world of a person, his thoughts, feelings in their movement and development.
• The plot and composition of the poem are considered. The plot in a lyric work is very often absent. This is mainly due to the fact that lyrics are expressive, not narrative; they express, not narrate.
• Description of the lyric hero and the transmission of the general mood of the poem. Life in a poetic text is presented through the experiences of the lyric hero. The concept of a lyrical hero is key for this type of literature. A lyrical hero is a bearer of emotion, feeling, experience, a certain state, on whose behalf a poem is created.

The lyrical hero often coincides with the author, but not always. For example, a poet can write a poem on behalf of a vagabond, a robber or a criminal – but this does not mean that the poet himself is a vagrant or a robber.

• Analysis of the vocabulary of the poem and means of expression. Expressive means make the verse brighter and richer. These include epithets, metaphors, allegories, impersonation, hyperbole, etc. You should also pay attention to the poetic meter, which the poem was written with.

There are the following poetic sizes: iambic, trochee, anapest, dactyl, amphibrachium

• At the end of the analysis, you need to express your own opinion about what you read, tell what feelings this work aroused.

Literature review

Analysis of the text is a short retelling, analysis of the content of any work. You can analyze poems, stories, poems, and so on. This teaches meaningful reading. That is why teachers give students such a task. Analysis of simple works helps to understand more complex works without problems. In addition, it develops the student's creativity, thinking, speech. The analysis helps to understand the meaning of the work, to highlight the moral and the main idea of ​​the author.

How to analyze a text based on literature:

It is necessary to re-read the author's work (with feeling, expression, pauses, changing intonations).

1. It is necessary to remember everything that is known about the author, or learn about his biography, interests from additional literature.
2. It is necessary to determine the style in which the work is written. There are only five of them: scientific, official-business, journalistic, artistic, colloquial.
3. After that, you need to determine the type of speech. There are only three of them: description, narration, reasoning.
4. It is necessary to indicate the genre of the work. Examples: play, story, ballad, epic, sketch, essay, and so on.
5. Then you need to determine the main theme of the work.
6. Write a headline or think about a headline. Why did the author choose this particular option?
7. Draw up a work plan, dividing it into semantic parts.
8. Identify ways of linking parts, pay special attention to lexical and syntactic means.
9. Determine the relationship, the ratio of the beginning and end of work.
10. Determine the basic literary techniques on which the text is built. Example: exaggeration, opposition, quick action changes, and so on.
11. Highlight and describe the main images, including the image of the author.
12. Study the phonetics of the text. What does the writer use and for what? Example: repetition of consonants or vowels.
13. Examine the morphological composition of the text. What parts of speech does the author use and for what?
14. Learn the syntax. Why did the author use these particular forms and for what?
15. Outline the main idea of ​​the text. What did the author want to say?
16. Express your personal impression of what you read, your feelings, emotions, states.

Special attention should be paid to the lexical analysis of the text. It is necessary to consider whether all the words are clear. If not, then you should refer to dictionaries.

What else to look for when analyzing the vocabulary of the text:

• keywords in each semantic part;
• supporting synonyms and antonyms;
• ambiguous words and words with a figurative meaning;
• the use of archaisms, historicisms, neologisms;
• colloquial expressions and sublime syllable;
• emotional and evaluative statements;
• phraseological units.

It is important to analyze separately the means of artistic expression. At each point, you need to answer the question: “Why, for what purpose did the author use this particular word?”

Universal plan

There are different variations of the plan. Some institutions allow for free presentation and classify this task as creative.

An abbreviated version of the plan for analysis:

Type of work: epic, lyric or drama.

1. Genre.
2. Several characteristics-features of the genre. For example: an adventure novel.
3. What inspired the author to create the plot. From my personal life, from the past, based on the works of other writers, and so on.
4. The main theme of the work. The theme is what is described in the work.
5. The main idea of ​​the work. The idea is the meaning of what the author wanted to say.
6. The main contradiction and its features (if any).
7. Composition.

If you fill in each item in detail, you get a detailed portrait of the author's work.

Comprehensive analysis involves parsing the text into three levels: ideas and images, text style, text phonetics (for lyric works).

How is text analysis done:

Reading text and dividing it into parts.

1. Title overview (topic, idea, meaning, problem).
2. Search and analysis of the author's position.
3. Search and analysis of micro themes.
4. Development of a text plan.
5. Parsing vocabulary, searching for definitions of unfamiliar words.
6. Studying information about the author (in what era he lived, what he was interested in, in what conditions he wrote the work).
7. Descriptions of the genre and composition.
8. Analysis of artistic means of expression.
9. Reader's attitude and impression.

When analyzing, it is important to pay attention to the system of images, their connection with each other and to the peculiarities of the development of the plot. Determine the principle of development, the outset, the main part, the culmination and the denouement.

3) Analysis of advertising channels

When analyzing advertising channels, it is very important to understand;

● what channels your competitors use to place their ads

● what target audience attracts each of them

The most important thing, when analyzing competitors, is to understand where potential consumers of our product or service come from to competitors. To see what our adversary is using to attract and retain a customer, you can use:

You can also check for the most frequent queries in the search engines Google, Yandex, Mail.ru, advertisements posted by our “dear friend”. To roughly calculate the amount of investment in advertising, open an advertising account (Yandex Direct, Google Adwords, Instagram, Google Merchant). Using these tools, you can see the cost of placement and can calculate your ad budget.

Analysis of advertising channels helps to find out what platforms a competitor uses to attract a customer, and whether you are ready to fight for the customer's attention by paying for advertising on these channels, or is it better to go where the competitor does not place his advertising.

4) Product economics

● In this market situation, can we stay afloat?

● How much does a competitor earn from customers?

● Will we be able to refocus the client on our product?

It is important to calculate the economics of your product or service before you start fighting for the consumer. What makes us profitable? How much does each attracted client cost us, and is it possible to entice a client from competitors?

Only in this way will we understand whether our company will stay afloat and with what result we will get out of this struggle. Earlier, we looked at what customer acquisition channels our competitor uses. Study what the economics of your opponents in the marketplace is and roughly estimate their advertising costs. If their business is growing and gaining more market share, then they are on the right track, but how to replicate and even surpass their success?

Calculate the profit and loss per client.

● Lifetime Value – what total profit the company receives from one client for the entire period of cooperation.

● Customer acquisition cost – the amount we pay to attract a new customer.

● Gross profit – the total income that we receive from all clients minus costs

● Return on investment – at this stage, the return on investment in advertising.

It is necessary to entice the consumer only when it is profitable. Sometimes it is easier and more profitable to beat off a client from a competitor than to spend money on finding a new one. Everything must be calculated. And it is important not to forget that market size also has an impact. Maybe there are such potential customers who have not yet been touched by advertising, and we can attract them? Or are customers involved in the market and the competition is fierce?

What is a complete blood count for? Why is this analysis so important?

Blood is a special tissue that is a transport for various substances between other tissues, organs and systems, while ensuring the unity and constancy of the internal environment of the body. Thus, most of the processes affecting the state of various tissues and organs, in one way or another, are reflected in the state of the blood.

Blood consists of plasma (liquid part of blood) and corpuscles – leukocytes, platelets, erythrocytes. Each type of corpuscular element has its own functions: leukocytes are responsible for immune protection, platelets – for blood coagulation, erythrocytes provide the transport of oxygen and carbon dioxide.

In a healthy person, the composition of the blood is quite constant, but with a disease it changes. Therefore, with the help of a blood test, it can be established that the disease is present. Sometimes a complete blood count can detect the disease at an early stage, when the main symptoms of the disease have not yet been manifested. That is why the UAC is carried out during any preventive examination. In the presence of symptoms, clinical analysis helps to understand the nature of the disease, to determine the intensity of the inflammatory process. Clinical analysis is used to diagnose various inflammatory diseases, allergic conditions, and blood diseases. A repeated general blood test will give the doctor the opportunity to judge the effectiveness of the prescribed treatment, assess the tendency to recovery and, if necessary, adjust the course of treatment.

Metric selection and validation procedure

The metric of the prediction quality of a (fuzzy) algorithm is a way to assess the quality of its work, to compare the result of its application with the actual answer. More mathematically, this is a function that takes as input a list of predictions

and a list of the answers that happened

, and returning a number corresponding to the prediction quality. For example, in the case of a classification problem, the simplest and most popular option is the number of mismatches

, and in the case of a regression problem – the standard deviation

… However, in some cases, for practical reasons, it is necessary to use less standard quality metrics.
Before introducing an algorithm into a product that is working and interacting with real users (or transferring it to a customer), it would be good to evaluate how well this algorithm works. For this, the following mechanism is used, called a validation procedure. The available labeled sample is divided into two parts – training and validation. The algorithm is trained on a training set, and its quality assessment (or validation) is on a validation set. In the event that we do not use a machine learning algorithm yet, but select a heuristic, we can assume that the entire labeled sample on which we evaluate the performance of the algorithm is validation, and the training sample is empty – it consists of 0 elements.

5) Risks and limitations

● What subtleties of the market should you pay attention to?

Depending on where you want to sell your product / service, different market nuances will arise. And with them risks will arise.

When selling fans in Norilsk, you should take into account that the temperature in these areas usually does not rise above 20 degrees.

The main aspects to focus on are:

● At what time does the client behave more actively (go back to the “Client portrait” section and add the necessary information to it)?

● What is the competitive environment in the selected territory?

● Are there any difficulties in selling goods in this region? Is there a representative office or do you need to add delivery to your list of services?

● Weather conditions.

● Currency exchange rate if fluctuations affect the value of the product.

● The way of life of the population, traditions.

Classic competitor analysis is based on SWOT analysis. The advantages of SWOT analysis are that it allows you to look quite simply, in the correct context, at the position of a company, product or service in the industry, to identify strengths and weaknesses, opportunities and threats, and therefore is the most popular tool in risk management and management decision-making. …

From my own experience, I will say that I received a lot of useful information that helped me develop my product by conducting a detailed SWOT analysis.

Sources used and useful links on the topic: https://habr.com/ru/post/352812/ https://FB.ru/article/330292/kak-delat-analiz-teksta-plan-i-shagi https: / /obrazovaka.ru/literatura/kak-delat-analiz-stihotvoreniya.html https://nauka.club/pomoshch-studentu/analiz-teksta.html https://vc.ru/marketing/46159-kak-provesti-analiz -konkurentov-svoimi-rukami https://www.fdoctor.ru/diagnostika/obshchiy_analiz_krovi/