Category Archives: System

How to adjust your Windows 10 volume settings so virtual meeting participants are at the same volume level

December 5, 2020System, Windows Administrationadjust meeting volume, can't hear person in meeting, chat, chat too loud, guide, how make person in virtual meeting louder, how make person in virtual meeting quieter, how set chat volume to same level, how set meeting volume to same level, How To, How to adjust your Windows 10 volume settings so virtual meeting participants are at the same volume level, lesson, meeting audio not at same level, meeting noise gate, meeting sound normalization, online meeting too loud, people too loud in chat, tutorialTidbytez

You have probably experienced a virtual meeting or online lecture were you could barely hear a person when they spoke. It might have been down to their audio setup, e.g. they were not using a headset and were relying on their laptop built-in microphone, or they might just be especially quiet when they talk. So in an effort to hear them better you maxed out the volume on your PC only for someone else in the chat to chime in with an excessively loud question or comment. If you were wearing a headset in that situation you might have ended up with burst eardrums. Thankfully there is something you can do on your end to normalize the volume of the meeting.

(FYI it is not recommended you test this for the first time during an important business meeting. Sound card drivers etc. can be a bit unpredictable and can even behave differently depending on what other applications you may have open at the time.)

What is Audio Normalization

Audio normalization is a process that increases the audio level by a constant amount so that it reaches a target or norm. Normalization applies the same level increase for the entire duration of the audio stream.

You are probably thinking at this point that if normalization brings up the volume of the quiet person wouldn’t it also make the loud person louder?

That is not the case. It averages out the loudness by leveling the audio output. So when the louder person interjects they should be at a similar volume to the person speaking quietly.

Normalizing the meeting volume

This is achieved through Windows 10 Sound settings and a feature called “Loudness equalization” and there are two ways to turn this feature on.

The first way is via the search bar at the bottom of your display.

Type “Sound settings”

Click on the option that appears.

When the Sound settings window opens look for “Device Properties” under “Output”, i.e. speakers/headphones etc.

In “Device Properties” click on “Additional device properties” to the right hand side of the window.

Open the “Enhancements” tab.

Make sure “Immediate mode” is ticked then scroll down to “Loudness equalization”.

Once the “Loudness equalization” box has been ticked you should hear that the volume of people speaking quietly has been increased. This means you will be able to lower down the overall system volume and when other people speak you should be able to hear them also without it being deafening.

The other way to get to this feature is the old fashion way via Control Panel.

Control Panel > Hardware and Sound > Sound > Left click on Speakers (or another output device of your choosing) > Properties > Enhancements > Check “Loudness equalization”.

If you found this post helpful please like/share/subscribe.

IT Project Management Failure: 3 Proposed Causes

November 2, 2020IT, Project Management, System3 reasons IT Projects fail, IT Projectment Failure, Management Level causes for project failures, Project Failure Causes, Project Management Failure Causes, Why do IT Projects fail, Why IT Projects FailTidbytez

Introduction

This article first highlights the misuse of the Project Management Triangle as a metric of success. Recognising that the very term “success”, and “failure”, can be subjective the author instead proposes generalised, objective and unambiguous examples of failure as a starting reference point. With these examples of failure serving as a foundation, three general deficits in project management are proposed as potential root causes, for IT project failure.

Project Management Triangle Misuse

The Project Management Triangle (also called the triple constraint, iron triangle and project triangle) consists of three points; cost, time and scope (or features). These points are argued to have proportionate relationships with each other. For example, a project can be completed faster by increasing budget and/or cutting scope. Similarly, increasing scope may require increasing the budget and/or schedule. Lowering the budget available will impact on schedule and/or scope. These trade-offs between the cost, time and scope create constraints which are said to dictate the quality of the produce. However stakeholders often misconstrue staying within the constraints of the triangle, while delivering a project, as a measure of success instead of, as intended, a determinant of quality.

As a demonstration of the unsuitability of the triangle as a metric of success consider the following. Would a self-build home delivered over budget, behind schedule and outside the original specifications be considered a failure? No, not for those who took on such a daunting endeavour, and survived the process, having brought into existence the home of their dreams. This is an example of a project where Atkinson (1999) might suggest the criteria for success existed outside of cost, time and scope.

So to define three significant causes of project failure it is first necessary to settle on unarguable features of project failure. It is important to note at this point that a project must have navigable obstacles and manageable risks. For instance an IT project cannot be considered a failure if an unnavigable obstacle was introduced, an example being new laws that prohibit online gambling that scuttle an online gambling platform that was in development. Similarly an IT project cannot be considered a failure if unmanageable risks were encountered such as the parent company collapsing due to financial irregularities not connected to the project.

With those points in mind the following statements are proposed as clear examples of project failure:

The project exhausted necessary resources with no or unfinished deliverables.
Delivery was too late and the deliverables are no longer needed or soon to be obsolete.
Deliverables are not fit for purpose or of relative value.
The costs exceeded the relative value generated by the deliverables.
The project killed the parent organisation.

With examples of failure defined above the following section proposes management level causes of IT project failures.

IT Project Failure: Management Level Causes

Poor Project Visibility

There is a recognised need to have an information system in place to report on progress, cost, schedule etc. (Larson and Gray, 2010) In the built environment progress can be apparent even to the eye of a lay person but visibility of progress and consumption of resources can be far more difficult for projects in other industries some of which have intangible deliverables. In the IT industry back end infrastructure projects for example may have no visible deliverables and with cloud based deployments no visible supporting hardware.

This is why project management styles like SCRUM and visualisation tools like Kanban boards and burn down charts have been adopted. Without these visualisation aids Project Managers could be blind to progress and resource consumption. Therefore a lack of visibility is proposed as a potential cause, or contributor, to any of the failure examples defined above.

Inadequate Domain Knowledge

Domain Knowledge Is vital in steering stakeholder specifications, knowing what the relevant mile stones are and establishing what is feasible given the budget, time and scope. The case is made by (Larson and Gray, 2010) that the key to managing scope creep, which can be beneficial, is change management. It is questioned however without adequate domain knowledge how can the project manager know what the knock-on effects of a change will be, the derived value of a change or even if a change is possible without putting the project at risk? It is also questioned if a lack of domain knowledge is often misunderstood as poor leadership?

Lack of Accountability

Accountability is seen by (Kerzner and Kerzner, 2017) as the combination of authority and responsibility that rests at an individual level and is necessary for work to move forward. It is argued that if team members are not assigned tasks with consequences for under performance or failure the project has no drive for completion. This was particularly evident in the PPARS project (“PPARS- a comedy of errors,” n.d.). Due to questionable contract arrangements there were strong financial incentives to not finish the project and without accountability driving the project forward that end result was a complete failure.

Conclusion

An IT Project Manager needs to utilise the project management triangle as intended i.e. a means to keep the desired level of quality of the deliverable in focus. It there are fluctuations in cost, time or scope the IT Project Manager needs to be cognizant of what the knock-on effects will be. In addition an IT Project Manager needs to know who the right person to assign specific tasks to is. That person needs to have the proper motivation to get the work done, with the IT Project Manager having visibility of the work being done and the knowledge and experience to be able to assess if the work is being done properly. This is achieved through individual accountability, project visibility and domain knowledge. Without these three elements it is proposed a project has little chance of success.

References:

Atkinson, R., 1999. Project management: cost, time and quality, two best guesses and a phenomenon, its time to accept other success criteria. International Journal of Project Management 17, 337–342. https://doi.org/10.1016/S0263-7863(98)00069-6

Kerzner, H., Kerzner, H.R., 2017. Project Management: A Systems Approach to Planning, Scheduling, and Controlling. John Wiley & Sons.

Larson, E.W., Gray, C.F., 2010. Project Management: The Managerial Process. McGraw-Hill Irwin.

NoClip, 2017. FINAL FANTASY XIV Documentary Part #1 – “One Point O” – YouTube [WWW Document]. URL https://www.youtube.com/watch?v=Xs0yQKI7Yw4 (accessed 10.7.20).

Pinto, J.K., Mantel, S.J., 1990. The causes of project failure. IEEE Transactions on Engineering Management 37, 269–276. https://doi.org/10.1109/17.62322

PPARS- a comedy of errors [WWW Document], n.d. URL http://www.irishhealth.com/article.html?id=8661 (accessed 10.13.18).

Do you need a Data Lake?

October 2, 2020Data, System, System architecturedata lake cloud, data lake problems, data lake vs data warehouse, how does a data lake work, should i use a data lake, What is a data lake, what is a data warehouse, when not to use a data lake, when not to use a data warehouse, when to use a data lake, why you don'd need a data lakeTidbytez

Summary

Among data specialist that do not work in the field of Big Data there can be confusion surrounding the term Data Lake. This is because there is apparent overlap in terms of role and function between Data Lakes and, the more traditional, Data Warehouses the likes of which data professionals will be more familiar with. This confusion is not helped by the term Data Lake itself being overloaded which will be discussed later in this article. However despite this overlap Data Lakes do occupy their own distinct role and perform functions Data Warehouses cannot.

Data Lakes have tremendous utility but damagingly there is also a mass of literature surrounding Data Lakes pushing the concept as a cure-all that coincidentally will also require you to migrate your organizations Business Intelligence center into the cloud. The following statements will hopefully dispel some of the associated hucksterism.

Data Lakes are not Data Warehouses 2.0, i.e. they are not the evolution of a Data Warehouse.
Data Lakes have not replaced Data Warehouses in performing the role of housing aggregated data.
Data Lakes will not free you from the burden of developing ETLs or establishing robust Data Architecture and strong Data Governance.

Introduction

It is important to first clarify that both Data Warehouses and Data Lakes are abstract concepts independent of any particular software or vendor. A Data Warehouse can be created in any database engine such as SQL Server, PostgreSQL, Oracle or MySql. Similarly a Data Lake can be deployed across any suitably large data storage platform, i.e. an on-site data center or hosted in the cloud.

In basic terms both Data Warehouses and Data Lakes can be thought of as the place where all data relevant to an organization’s goals is pulled together from various sources both internal and external (increasingly external). They both exist to facilitate an all encompassing view of an organization and how will it performs or provide a greater understanding of the organization’s environment, opportunities (e.g. customer preferences and habits) and threats. However they differ in terms of the data they are optimized to handle and are therefore better suited to different use cases.

What is a Data Warehouse?

A Data Warehouse is a method for storing and organising data that is optimized to support Business Intelligence (BI) activities such as analytics. To put it another way they solely exist and are constructed in a manner to best answer big questions efficiently. For this reason they typically hold vast quantities of historical data. The data within a data warehouse is usually derived from a wide range of sources such as application log files but primarily transaction applications (Oracle, 2019). However in contrast to a transactional database were each transaction is a separate record, the relevant entries in a Data Warehouse are typically aggregated although they can also hold transaction records for archival purposes.

Figure 1: Typical Data Warehouse architecture of an SME (Databricks, 2019)

Single transaction records on their own are not typically very insightful to an organization, trying to identify consumer trends for example. Aggregating data based on facts and dimensions, e.g. the number of sales (fact) for a particular store (dimension), saves disk space and allows queries looking for that specific answer to be returned quickly. Data Warehouses mostly contain numeric data which is easily manipulated. As an example store sales might be the summation of thousands of rows of data to a single row.

Figure 2: Simplified example of a Data Warehouse internal structure (BIDataPro, 2018)

Data Warehouses also solve the problem of trying to derive information when there are too many sources, e.g. a multinational with thousands of store locations and subsidiaries, by creating a “single source of truth”. Effectively this means pulling all the information to one central location, transforming the data for uniformity and storing like for like data together. For example this could mean gathering all sales data from multiple locations and converting the currency to dollars. All of the data in one place together allows for different sources, which serve different purposes, to be combined via a single query. For example a report that links sales data and logistical data, coming from POS and SCM systems respectively, may not be possible with a single query if the systems are not linked. If best practices regarding I.T. security are being followed they certainly should not be.

Data Warehouses are fed from source systems using an extract, transform and load (ETL) solution. This means data is extracted from a source system, transformed to meet schema and business requirements of the Data Warehouse and then loaded. This is a data delivery method independent of any particular software vendor. There are various software to accomplish ETLs including the option to create a custom application. A variation of this process is extract, load and transform (ELT) in which the data is landed into tables raw and later transformed to meet the schema and business requirements of their intended final table. This method allows for greater auditability which could aid in regulatory compliance or post-mortems if the transformation process fails.

Once set up the Data Warehouse can facilitate statistical analysis, reporting, data mining and more sophisticated analytical applications that generate actionable information by applying machine learning and artificial intelligence (AI) algorithms (Oracle, 2019).

For an organization a single source of truth which will eliminate inconsistencies in reporting, establish a single set of global metrics and allow everyone in the organization to “sing from the same hymn sheet” is very important due to how beneficial the information provided is in directing informed decisions.

So if Data Warehouses have proven such an excellent platform for generating information why are alternatives needed? Well by design only a subset of the attributes are examined, so only pre-determined questions can be answered (Dixon, 2010). Also the data is aggregated so visibility into the lowest levels is lost (Dixon, 2010). The final major factor is that some of the most vital sources of information are no longer simply numerical in nature and generated by an organizations internal transactional system. So what has changed?

The Digital Universe

The data landscape has changed drastically in just a few short years. Like the physical universe, the digital universe is large and growing fast. It was estimated that by 2020 there would be nearly as many digital bits as there are stars in the observable universe (Turner, 2014). That estimate is somewhere in the region of 44 zettabytes, or 44 trillion gigabytes (Turner, 2014). Even though this quantity of data is already beyond human comprehension the rate of growth is probably the more impressive feat. For context there is over 10 times more data now than there was in 2013 when the digital universe was an estimated 4.4 zettabytes (Turner, 2014). The data we create and copy annually is estimated to reach 175 zettabytes by 2025 (Coughlin, 2018).

Where is all this data coming from?

The short answer is predominately us and the systems that service our needs. In the not too distant past the only entities to have computers generating and storing data were businesses, governments and other institutions. Now everyone has a computer of some description and with the advent of social media mass consumers became mass creators. When you stop to think of how many interactions a person has with electronic devices every day, directly or indirectly, you soon get a picture of how much data is actually being generated.

As an example of this endless generation of data the following are average social media usage stats over the course of one minute from 2018 (Marr, 2018):

Twitter users sent 473,400 tweets
Snapchat users shared 2 million photos
Instagram users posted 49,380 pictures
LinkedIn gained 120 new users

Other extraordinary data stats include (Marr, 2018):

Google processes more than 40,000 searches every second or 3.5 billion searches a day.
5 billion people are active on Facebook every day. That’s one-fifth of the world’s population.
Two-thirds of the world’s population now owns a mobile phone.

Our way of life has become increasingly digitized with no better example than the effective global lockdown during the 2020 pandemic. Hundreds of millions of employees from around the world managed to continue working from home and did so effectively (Earley, 2020). This would have been unimaginable even by the late nineties. And yet as digitized as our world has become it is only the start. With emerging technologies such as self-driving cars, IoT smart devices and ever increasingly sophisticated robots entering our homes the 175 zettabytes of data by 2025 maybe a conservative estimate.

With so much of the stuff you would be forgiven for thinking all of this data is just a by-product but it is anything but. The data generated is an incredibly valuable asset if it can be analyzed properly and transformed into business relevant information.

What types of data are there?

The state of data within the digital universe can be summarized as structured, semi-structured and unstructured (Hammer, 2018).

The following is a non-exhaustive list of data types (Hammer, 2018):

CRM
POS
Financial
Loyalty card
Incident ticket
Email
PDF
Spreadsheet
Word processing
GPS
Log
Images
Social media
XML/JSON
Click stream
Forums
Blogs
Web content
RSS feed
Audio
Transcripts

Only the data types above in bold are suitable for aggregation (Hammer, 2018). The rest of the data types are typical of what now makes up a large proportion of the digital universe, and despite their value as data assets they are not suitable for storage or analysis within a Data Warehouse. This is because data needs to meet the predefined structure of a Data Warehouse in order for it to be accepted and aggregating these raw unstructured files, e.g. video and audio files etc., is not possible. So how are these types of valuable data turned into actionable information?

What is a Data Lake?

Data Warehouses have been utilized by data specialists for decades but the concept of Data Lakes is much more contemporary and much better suited to deal with storage, analysis and analytics of the semi-structured and unstructured data listed above. By design storage within a Data Lake of these kinds of data does not require files to be transformed as the file is kept in a raw state. Files can be simply copied from one file structure to another. Data Lakes also allow for working off the files directly which means the data can be used effectively immediately, i.e. as soon as it lands, rather than waiting weeks for the Data Warehouse developers to massage the data into a format that the data warehouse can accept if that is even possible (Hammer, 2018). Working with this type of data has become synonymous with the field of Big Data, which is defined by high velocity, high volume and high variability. As such the two methodologies of Data Warehouses and Data Lakes are not necessarily in competition with each other either, in fact depending on their definition (Data Lake is somewhat of an overloaded term (Bethke, 2017)) they could be argued to resolve difference problems and can complement each other when deployed within the same architecture.

There is some contention as to the definition of a Data Lake. Some would argue that original meaning implied the Lake was a raw data reservoir solely (Bethke, 2017). By this definition the Data Lake is not too dissimilar to a staging area or Operational Data Store (ODS) in a data warehouse were raw copies of data from source systems are landed (Bethke, 2017). This would coincide with an ELT process as opposed to an ETL process. The transform and integration of the data happens later downstream during the populating of the data warehouse (Bethke, 2017). This understanding of a Data Lake still persists today in the minds of many data specialist as can be seen below in the overly simplified illustration.

Figure 3: Overly simplified illustration of a Data Lake architecture (Hammer, 2018)

(Note: no indication of analysis being performed on the lake directly, the lake services the warehouse solely)

However it is an inaccurate understanding as the person who is credited with coining the term, James Dixon, used the following analogy when he explained a Data Lake:

“If you think of a datamart as a store of bottled water – cleansed and packaged and structured for easy consumption – the data lake is a large body of water in a more natural state. The contents of the data lake stream in from a source to fill the lake, and various users of the lake can come to examine, dive in, or take samples.” (Dixon, 2010)

By stating “various users of the lake can come to examine, dive in, or take samples” Dixon is clearly implying that a feature of the Data Lake is that it is accessible prior to the data being transformed and made available in a Data Warehouse.

This is where Data Lakes and Data Warehouses take an opposing strategy on applying structure to data which is perhaps why they are often mistaken as alternative competing concepts to each other. A Data Warehouse requires Schema on Write whereas a Data Lake uses Schema on Read.

With schema on write all of the relevant data structure needs to be prepared in advance which means all of the relevant business questions need to be thought of in advance. This rarely results in a situation where all the relevant stakeholders have their needs met and if they do it will not be for very long. This scenario is workable by an organization looking to aggregate finance data they are very familiar with but it is especially difficult when dealing with Big Data were the questions are unknown.

With schema on read the schema is only applied when the data is read allowing for a schema that is adaptable to the queries being issued. This means you are not stuck with a predetermined one-size-fits-all schema (Pasqua, 2014). This allows for the storage of unstructured data and since it is not necessary to define the schema before storing the data it makes it easier to bring in new data sources on the fly. The exploding growth of unstructured data and overhead of ETL for storing data in RDBMS is the offered as a leading reason for the shift to schema on read (Henson, 2016).

When dealing with Big Data the problem of a predefined schema can be so burdensome that it can sink a data project or increase the time-to-value past the point of relevance (Pasqua, 2014). Using a schema on read approach on data as-is means getting value from it right away (Pasqua, 2014). The flexibility of Data Lakes in this regard allows them to surpass Data Warehouses in terms of scalability while making data accessible for analysis sooner.

Data Lakes Scalability

By using schema on read the constraint on scale is virtually removed. The threat of a bottleneck still exists but now in the form of physical constraints in terms of the hardware available. This is why online cloud offerings such as Amazon S3 and Azure Data Lake from Microsoft have become so popular. Of course on-site data centers are also an option with Hadoop being a very popular solution which combines a Data Lake structure with analytically capabilities. This level of scalability also safe guards against Data Silos. A Data Silo is an undesirable situation where only one group or a limited number of people in an organization have access to a source of data that has a broader relevance to people across an organization (Plixer, 2018).

Data Lakes are intended by design and philosophy to be an antithesis to Data Silos where all an organizations data is stored together in one lake. However centrally storing all data is not without significant security concerns and losing sight of what customer data is on hand can run afoul of numerous legal requirements such as GDPR.

Data Lakes Analysis & Analytics

A defining feature of Big Data analytics is the concept of bringing the analytics to the data rather than the data to the analytics. Traditionally analytics was carried out by feeding single flat files into an algorithm with the time taken to prepare these files being significant. Although accessing the raw files directly is potentially a failing as it has the potential to break the principle of a single source of truth and therefore runs the risk of introducing inconsistencies between reports and other forms of analysis. As you can imagine this is complex and disciplined work which is why Data Lakes, at this point in their maturity, are best suited to Data Scientists and advanced Data Analysts (Hammer, 2018). However this goes against the Data Lake ethos of “data for all” as it only allows the very skilled to have access. This creates the problem Data Lakes were meant to solve by imposing restrictions or “data for the select few”. With Data scientists acting as the gatekeepers an organizations stakeholders can lose sight of the useful data available to them. Worse still is that valuable data may come from external sources with stakeholders having no visibly of it prior to it landing in the Data Lake. This may leave stakeholders with no option but to take action based on an analysis produced by a Data Scientist with accuracy of the analysis being a matter of fate because the stakeholder has no data to say otherwise. In comparison the creation of a Data Warehouse is usually a collaboration of stakeholders, familiar with internal sources systems and data, and developers. Once a Data Warehouse is created, far less skilled (and cheaper) Data Analysts will have the ability to navigate the internal structure and compile valuable reports.

Despite the obvious concerns the significance of scalability and direct raw data analysis cannot be overlooked. The sooner an organization is informed the sooner it can act. In real world terms this could save millions of dollars, save thousands of jobs or stop the organizations itself from going under. However the benefits of scalability and earlier data access are not without risks as poorly managed Data Lakes have the potential to turn into Data Swamps. Data Swamps are poorly managed Data Lakes that become a dumping ground for data. Though the data may be unstructured the method in which it is stored must not be or visibility of what is stored and where it is stored will be lost. Failure to catalogue the data, letting users know what is available while making the attributes of the data known, will overwhelm users and result in the garbage results (Hammer, 2018). Successful implementation of a Data Lake is complex and requires ongoing commitment to maintain but for a large organization that needs to make better use of the wider range of data available in the digital universe a Data Lake is a necessity.

Conclusion

A Data Lake is not a replacement for a Data Warehouse. Data Lakes are better equipped to solve the different problems associated with dealing with semi-structured to unstructured data. Their flexibility in this regard allows them to surpass Data Warehouses in terms of scalability while making data accessible for analysis sooner. However Data Lakes are not without their drawbacks. They require highly skilled and expensive staff to develop and maintain. They potentially run a greater risk of failing spectacularly by devolving into a Data Swamp and could potentially become a serious liability from a regulatory standpoint if this was to happen. Organisations can also be left at the mercy of Data Scientists in how accurate they are in analyzing data and producing correct reports as stakeholders may not have the expertise to retrieve data from the Data Lake themselves.

Thankfully Data Warehouses are still perfectly suited for dealing with numeric data and for organizations that still predominately use their own internal transactional systems in the creation of actionable information these organisations have no immediate need to utilize any alternatives.

References:

Bethke, U. (2017) ‘Are Data Lakes Fake News?’, Sonra, 8 August. Available at: http://www.kdnuggets.com/2017/09/data-lakes-fake-news.html (Accessed: 4 July 2020).

BIDataPro (2018) ‘What is Fact Table in Data Warehouse’, BIDataPro, 23 April. Available at: https://bidatapro.net/2018/04/23/what-is-fact-table-in-data-warehouse/ (Accessed: 4 July 2020).

Coughlin, T. (2018) 175 Zettabytes By 2025, Forbes. Available at: https://www.forbes.com/sites/tomcoughlin/2018/11/27/175-zettabytes-by-2025/ (Accessed: 4 July 2020).

Databricks (2019) ‘Unified Data Warehouse’, Databricks, 8 February. Available at: https://databricks.com/glossary/unified-data-warehouse (Accessed: 4 July 2020).

Dixon, J. (2010) ‘Pentaho, Hadoop, and Data Lakes’, James Dixon’s Blog, 14 October. Available at: https://jamesdixon.wordpress.com/2010/10/14/pentaho-hadoop-and-data-lakes/ (Accessed: 4 July 2020).

Earley, K. (2020) Google and Facebook extend work-from-home policies to 2021, Silicon Republic. Available at: https://www.siliconrepublic.com/companies/google-facebook-remote-work-until-2021 (Accessed: 5 July 2020).

Hammer, D. (2018) What is a data lake? – The Hammer | The Hammer. Available at: https://www.sqlhammer.com/what-is-a-data-lake/ (Accessed: 4 July 2020).

Henson, T. (2016) ‘Schema On Read vs. Schema On Write Explained’, Thomas Henson, 14 November. Available at: https://www.thomashenson.com/schema-read-vs-schema-write-explained/ (Accessed: 6 July 2020).

Marr, B. (2018) How Much Data Do We Create Every Day? The Mind-Blowing Stats Everyone Should Read, Forbes. Available at: https://www.forbes.com/sites/bernardmarr/2018/05/21/how-much-data-do-we-create-every-day-the-mind-blowing-stats-everyone-should-read/ (Accessed: 5 July 2020).

Oracle (2019) What Is a Data Warehouse | Oracle Ireland. Available at: https://www.oracle.com/ie/database/what-is-a-data-warehouse/ (Accessed: 5 July 2020).

Pasqua, J. (2014) Schema-on-Read vs Schema-on-Write, MarkLogic. Available at: https://www.marklogic.com/blog/schema-on-read-vs-schema-on-write/ (Accessed: 6 July 2020).

Plixer (2018) What is a Data Silo and Why is It Bad for Your Organization? Available at: https://www.plixer.com/blog/data-silo-what-is-it-why-is-it-bad/ (Accessed: 6 July 2020). Turner, V. (2014) The Digital Universe of Opportunities. Available at: https://www.emc.com/leadership/digital-universe/2014iview/executive-summary.htm (Accessed: 5 July 2020).

How to use a generic PC controller with GTA 5

August 2, 2020Gaming, Systemgeneric controller, GTA 5, GTA V, guide, How To, How to use a PC controller with GTA 5, howto, lesson, PC, PC controller not working GTA 5, tutorial, XBox controllerTidbytez 1

So you plugged in the cheap PC controller you bought off eBay or Amazon (say one that is coincidentally shaped like an Xbox controller) and found it doesn’t work with GTA 5?

This is probably because the controller is using the DirectInput standard as opposed to the newer XInput standard. You can read more about these standards from Microsoft by clinking on this link.

GTA 5 (or GTA V if you’re feeling fancy) was not optimised to use the DirectInput standard unfortunately. However if you’re playing the game on PC you should be using a keyboard and mouse like a grown-up anyway, it’s way better for shooters. Ah but GTA isn’t all about the shooting I hear you say and you’re right. I’ll admit I switch to a controller for flying vehciles because they are horrible to pilot with direction keys. Analogue sticks are much better suited to aircraft.

So the workaround for being able to use the cheap generic controller is quite simple but requires using an “Xbox 360 Controller Emulator”.

Download the file x360ce_x64 from the following site by clicking on the “download for 64 bit games” button at the top of the screen.

https://www.x360ce.com/

(Be sure to test the downloaded file with whatever antivirus software you have installed)

Once you are satisfied the software is safe extract the file to the root directory of where you installed GTA 5.

(If you do not know where that folder is try searching GTAV in your Windows search bar. Note a folder called GTA V is often created in the Documents folder but this is not the correct directory. The correct directory will have application files with the GTAV logo in it. The game maybe in a Rockstar folder or perhaps a Steam or Epic folder, it all depends on who you bought it from.)

Right click on x360ce_x64 and run the file as administrator and you should be given an option button to “Create”.

Click this button and a Xbox controller calibration window will open.

At this point if your controller has a large circular button at its center press it and make sure it lights up otherwise it may not send the right signals when the controller is being mapped.

Click “Auto” and then “Save”.

I found the A, B, X and Y face buttons were not mapped correctly and needed to be mapped manually.

To do this, beside each face button on the emulator interface there is a drop down menu. Click on it for each button and choose the option to record. The interface will highlight which button to press on your controller to map it correctly. Once the buttons are mapped correctly click “Save” again.

After following these steps you should now be able to play GTA V with the controller.

How I passed The CompTIA CySA+ exam

April 2, 2020IT, OS, security, Systemcomptia, cysa, cysa+, guide, How To, How to pass comptia cysa+ exam, howto, lesson, ports, tips to pass cysa, tutorialTidbytez

I did the exam back at the start of January 2020 and as someone who is not from a security or networking background I did find it difficult.

To put it bluntly there’s no point even attempting the exam without knowing ports, all the relevant security software (including the various common commands) and the IT security frameworks.

I found the Jason Dion Udemy course very helpful. (FYI not affiliated in anyway) The course however should only be considered supplementary. You should be doing your own research and researching the topics and terms within the course you think weren’t covered in as much detail as they could have been.

Most importantly of all do practice exams. There are several floating around on the internet and Jason Dion has an additional set of his own on Udemy.

As part of my research I prepared a list of definitions which are shared below. This is just a big dump of text but worth giving a few minutes of your time to freshen up on acronyms. (My apologies to whoever prepared the source material, I never took note where it came from for the sake of references.)

Internet Protocol (IP) is the principal set (or communications protocol) of digital message formats and rules for exchanging messages between computers across a single network or a series of interconnected networks, using the Internet Protocol Suite (often referred to as TCP/IP). Messages are exchanged as datagrams, also known as data packets or just packets.

IP is the primary protocol in the Internet Layer of the Internet Protocol Suite, which is a set of communications protocols consisting of four abstraction layers: link layer (lowest), Internet layer, transport layer and application layer (highest).

The main purpose and task of IP is the delivery of datagrams from the source host (source computer) to the destination host (receiving computer) based on their addresses. To achieve this, IP includes methods and structures for putting tags (address information, which is part of metadata) within datagrams.

Think of an anology with the postal system. IP is similar to the U.S. Postal System in that it allows a package (a datagram) to be addressed (encapsulation) and put into the system (the Internet) by the sender (source host). However, there is no direct link between sender and receiver.

The package (datagram) is almost always divided into pieces, but each piece contains the address of the receiver (destination host). Eventually, each piece arrives at the receiver, often by different routes and at different times. These routes and times are also determined by the Postal System, which is the IP.

However, the Postal System (in the transport and application layers) puts all the pieces back together before delivery to the receiver (destination host).

Note: IP is actually a connectionless protocol, meaning that the circuit to the receiver (destination host) does not need be set up before transmission (by the source host). Continuing the analogy, there does not need to be a direct connection between the physical return address on the letter/package and the recipient address before the letter/package is sent.

Originally, IP was a connectionless datagram service in a transmission control program created by Vint Cerf and Bob Kahn in 1974. When format and rules were applied to allow connections, the connection-oriented Transmission Control Protocol was created. The two together form the Internet Protocol Suite, often referred to as TCP/IP.

Internet Protocol version 4 (IPv4) was the first major version of IP. This is the dominant protocol of the Internet. However, iPv6 is active and in use, and its deployment is increasing all over the world.

Addressing and routing are the most complex aspects of IP. However, intelligence in the network is located at nodes (network interconnection points) in the form of routers which forward datagrams to the next known gateway on the route to the final destination. The routers use interior gateway protocols (IGPs) or external gateway protocols (EGPs) to help with making forwarding route decisions.

Routes are determined by the routing prefix within the datagrams. The routing process can therefore become complex. But at the speed of light (or nearly so) the routing intelligence determines the best route, and the datagram pieces and datagram all eventually arrive at their destination.

The process of putting these tags on datagrams is called encapsulation.

How TCP Works

Transmission Control Protocol (TCP) is the most commonly used protocol on the Internet.

When you request a web page in your browser, your computer sends TCP packets to the web server’s address, asking it to send the web page back to you. The web server responds by sending a stream of TCP packets, which your web browser stitches together to form the web page. When you click a link, sign in, post a comment, or do anything else, your web browser sends TCP packets to the server and the server sends TCP packets back.

TCP is all about reliability—packets sent with TCP are tracked so no data is lost or corrupted in transit. This is why file downloads don’t become corrupted even if there are network hiccups. Of course, if the recipient is completely offline, your computer will give up and you’ll see an error message saying it can’t communicate with the remote host.

TCP achieves this in two ways. First, it orders packets by numbering them. Second, it error-checks by having the recipient send a response back to the sender saying that it has received the message. If the sender doesn’t get a correct response, it can resend the packets to ensure the recipient receives them correctly.

User Datagram Protocol (UDP) is used by apps to deliver a faster stream of information by doing away with error-checking. When configuring some network hardware or software, you may need to know the difference.

How UDP Works

The UDP protocol works similarly to TCP, but it throws out all the error-checking stuff. All the back-and-forth communication introduces latency, slowing things down.

When an app uses UDP, packets are just sent to the recipient. The sender doesn’t wait to make sure the recipient received the packet—it just continues sending the next packets. If the recipient misses a few UDP packets here and there, they are just lost—the sender won’t resend them. Losing all this overhead means the devices can communicate more quickly.

UDP is used when speed is desirable and error correction isn’t necessary. For example, UDP is frequently used for live broadcasts and online games.

For example, let’s say you’re watching a live video stream, which are often broadcast using UDP instead of TCP. The server just sends a constant stream of UDP packets to computers watching. If you lose your connection for a few seconds, the video may freeze or get jumpy for a moment and then skip to the current bit of the broadcast. If you experience minor packet-loss, the video or audio may be distorted for a moment as the video continues to play without the missing data.

This works similarly in online games. If you miss some UDP packets, player characters may appear to teleport across the map as you receive the newer UDP packets. There’s no point in requesting the old packets if you missed them, as the game is continuing without you. All that matters is what’s happening right now on the game server—not what happened a few seconds ago. Ditching TCP’s error correction helps speed up the game connection and reduce latency.

Transport Layer Security (TLS) is a protocol that provides communication security between client/server applications that communicate with each other over the Internet. It enables privacy, integrity and protection for the data that’s transmitted between different nodes on the Internet. TLS is a successor to the secure socket layer (SSL) protocol.

TLS primarily enables secure Web browsing, applications access, data transfer and most Internet-based communication. It prevents the transmitted/transported data from being eavesdropped or tampered. TLS is used to secure Web browsers, Web servers, VPNs, database servers and more. TLS protocol consists of two different layers of sub-protocols:

TLS Handshake Protocol: Enables the client and server to authenticate each other and select a encryption algorithm prior to sending the data

TLS Record Protocol: It works on top of the standard TCP protocol to ensure that the created connection is secure and reliable. It also provides data encapsulation and data encryption services.

Port Descriptions:

File Transfer Protocol (FTP) is a standard network protocol used for the transfer of computer files between a client and server on a computer network.

FTP is built on a client-server model architecture using separate control and data connections between the client and the server.[1] FTP users may authenticate themselves with a clear-text sign-in protocol, normally in the form of a username and password, but can connect anonymously if the server is configured to allow it. For secure transmission that protects the username and password, and encrypts the content, FTP is often secured with SSL/TLS (FTPS) or replaced with SSH File Transfer Protocol (SFTP).

Ports:

Send File Data Port 20
Session Info 21

Secure Shell (SSH) is a cryptographic network protocol for operating network services securely over an unsecured network. Typical applications include remote command-line, login, and remote command execution, but any network service can be secured with SSH.

SSH provides a secure channel over an unsecured network in a client–server architecture, connecting an SSH client application with an SSH server. The protocol specification distinguishes between two major versions, referred to as SSH-1 and SSH-2. The standard TCP port for SSH is 22. SSH is generally used to access Unix-like operating systems, but it can also be used on Microsoft Windows. Windows 10 uses OpenSSH as its default SSH client. Port 22.

SSH File Transfer Protocol (SFTP) is a secure file transfer protocol. It runs over the SSH protocol. It supports the full security and authentication functionality of SSH.

SFTP has pretty much replaced legacy FTP as a file transfer protocol, and is quickly replacing FTP/S. It provides all the functionality offered by these protocols, but more securely and more reliably, with easier configuration. There is basically no reason to use the legacy protocols any more.

SFTP also protects against password sniffing and man-in-the-middle attacks. It protects the integrity of the data using encryption and cryptographic hash functions, and autenticates both the server and the user.

SFTP port number is the SSH port 22 (follow the link to see how it got that number). It is basically just an SSH server. Port 22.

Telnet is a computer protocol that provides two-way interactive communication compatibility for computers on the internet and local area networks. User data is interspersed in-band with Telnet control information in an 8-bit byte oriented data connection over the Transmission Control Protocol (TCP). Telnet has a command-line interface and is famous for being the original protocol from when the internet first launched in 1969. Where a telnet application is listening it operates on port 23.

Vulnerability of Telnet

In time, Telnet’s use declined in favor of SSH (Secure Shell or Secure Socket Shell) due to serious security concerns when it was used over an open network. Telnet lacks authentication policies and data encryption.

Simple Mail Transfer Protocol (SMTP) is an internet standard for email supported by most email processing servers. Port 25. SMTPS (Simple Mail Transfer Protocol Secure) is a method for securing the SMTP using transport layer security. It is intended to provide authentication of the communication partners, as well as data integrity and confidentiality. Port 465.

SMTPS is not a proprietary protocol and not an extension of SMTP. It is a way to secure SMTP at the transport layer, by wrapping SMTP inside TLS. Conceptually, it is similar to how HTTPS wraps HTTP inside TLS.

WHOIS (pronounced as the phrase “who is”) is a query and response protocol that is widely used for querying databases that store the registered users or assignees of an Internet resource, such as a domain name, an IP address block or an autonomous system, but is also used for a wider range of other information. The protocol stores and delivers database content in a human-readable format. Port 43.

Terminal Access Controller Access-Control System (TACACS) refers to a family of related protocols handling remote authentication and related services for networked access control through a centralized server. The original TACACS protocol, which dates back to 1984, was used for communicating with an authentication server, common in older UNIX networks; it spawned related protocols:

Extended TACACS (XTACACS) is a proprietary extension to TACACS introduced by Cisco Systems in 1990 without backwards compatibility to the original protocol. TACACS and XTACACS both allow a remote access server to communicate with an authentication server in order to determine if the user has access to the network.

Terminal Access Controller Access-Control System Plus (TACACS+) is a protocol developed by Cisco and released as an open standard beginning in 1993. Although derived from TACACS, TACACS+ is a separate protocol that handles authentication, authorization, and accounting (AAA) services. TACACS+ have largely replaced their predecessors. Port 49.

The Domain Name System (DNS) is the phonebook of the Internet. Humans access information online through domain names, like nytimes.com or espn.com. Web browsers interact through Internet Protocol (IP) addresses. DNS translates domain names to IP addresses so browsers can load Internet resources. Port 53.

Each device connected to the Internet has a unique IP address which other machines use to find the device. DNS servers eliminate the need for humans to memorize IP addresses such as 192.168.1.1 (in IPv4), or more complex newer alphanumeric IP addresses such as 2400:cb00:2048:1::c629:d7a2 (in IPv6).

Dynamic Host Configuration Protocol (DHCP) is a protocol that provides quick, automatic, and central management for the distribution of IP addresses within a network. DHCP is also used to configure the subnet mask, default gateway, and DNS server information on the device. Port 67.

Bootstrap Protocol (BOOTP) is a computer networking protocol used in Internet Protocol networks to automatically assign an IP address to network devices from a configuration server Port 67.

Differences between DHCP and BOOTP:

The major difference between BOOTP and DHCP is that BOOTP support static configuration of the IP addresses while DHCP supports dynamic configuration. It means that DHCP automatically assigns and obtain the IP addresses from the computer connected to the internet and also have some additional features.

https://techdifferences.com/difference-between-bootp-and-dhcp.html

Trivial File Transfer Protocol (TFTP) is a technology that transfers files between network devices and is a simplified version of the more robust File Transfer Protocol. TFTP was developed in the 1970s for computers lacking sufficient memory or disk space to provide full FTP support. Today, TFTP is found on consumer broadband routers and commercial network routers. Home network administrators use TFTP to upgrade the router firmware, while professional administrators use TFTP to distribute software across corporate networks. Port 69.

Hypertext Transfer Protocol (HTTP) is an application protocol for distributed, collaborative, hypermedia information systems.[1] HTTP is the foundation of data communication for the World Wide Web, where hypertext documents include hyperlinks to other resources that the user can easily access, for example by a mouse click or by tapping the screen in a web browser. Port 80.

Development of HTTP was initiated by Tim Berners-Lee at CERN in 1989. Development of early HTTP Requests for Comments (RFCs) was a coordinated effort by the Internet Engineering Task Force (IETF) and the World Wide Web Consortium (W3C), with work later moving to the IETF.

Hypertext Transfer Protocol Secure (HTTPS) is an extension of the Hypertext Transfer Protocol (HTTP). It is used for secure communication over a computer network, and is widely used on the Internet. In HTTPS, the communication protocol is encrypted using Transport Layer Security (TLS) or, formerly, its predecessor, Secure Sockets Layer (SSL). The protocol is therefore also often referred to as HTTP over TLS, or HTTP over SSL. Port 443.

Secure Socket Tunneling Protocol (SSTP) is a form of virtual private network (VPN) tunnel that provides a mechanism to transport PPP traffic through an SSL/TLS channel. SSL/TLS provides transport-level security with key negotiation, encryption and traffic integrity checking. The use of SSL/TLS over TCP port 443 allows SSTP to pass through virtually all firewalls and proxy servers except for authenticated web proxies. Port 443.

Kerberos is a computer-network authentication protocol that works on the basis of tickets to allow nodes communicating over a non-secure network to prove their identity to one another in a secure manner. Port 88.

Post Office Protocol version 3 (POP3) is a standard mail protocol used to receive emails from a remote server to a local email client. POP3 allows you to download email messages on your local computer and read them even when you are offline. Note, that when you use POP3 to connect to your email account, messages are downloaded locally and removed from the email server. This means that if you access your account from multiple locations, that may not be the best option for you. On the other hand, if you use POP3, your messages are stored on your local computer, which reduces the space your email account uses on your web server.

By default, the POP3 protocol works on two ports:

Port 110 – this is the default POP3 non-encrypted port

Port 995 – this is the port you need to use if you want to connect using POP3 securely

Network News Transfer Protocol (NNTP) is the underlying protocol of UseNet, which is a worldwide discussion system which contains posts or articles which are known as news. Network News Transfer Protocol is used to transfer news from one network to another. It is designed specifically to transfer news/articles. A NNTP client is included in browsers like Netscape, Opera and Internet Explorer or a special application named newsreader can be used as a NNTP client. NNTP servers manages the global network of news groups. Port 119.

Network time protocol (NTP) is a networking protocol for clock synchronization between computer systems over packet-switched, variable-latency data networks. In operation since before 1985, NTP is one of the oldest Internet protocols in current use. it has become a key component in security as when you are tracing through logs to pull together events from different devices, accurate time throughout your network is incredibly important. Port 123.

NetBIOS over TCP/IP (NBT, or sometimes NetBT) is a networking protocol that allows legacy computer applications relying on the NetBIOS API to be used on modern TCP/IP networks. Ports 137, 138, 139.

Internet Message Access Protocol (IMAP) is an Internet standard protocol used by email clients to retrieve email messages from a mail server over a TCP/IP connection. IMAP is defined by RFC 3501.

IMAP was designed with the goal of permitting complete management of an email box by multiple email clients, therefore clients generally leave messages on the server until the user explicitly deletes them. An IMAP server typically listens on port 143. IMAP over SSL (IMAPS) is assigned the port 993.

Simple Network Management Protocol (SNMP) is an Internet Standard protocol for collecting and organizing information about managed devices on IP networks and for modifying that information to change device behavior. Devices that typically support SNMP include cable modems, routers, switches, servers, workstations, printers, and more.

SNMP is widely used in network management for network monitoring. SNMP exposes management data in the form of variables on the managed systems organized in a management information base (MIB) which describe the system status and configuration. These variables can then be remotely queried (and, in some circumstances, manipulated) by managing applications.

Ports:

161 – SNMP (Agents receive requests)
162 – SNMP (Controller receives data)

Lightweight Directory Access Protocol (LDAP) is an open, vendor-neutral, industry standard application protocol for accessing and maintaining distributed directory information services over an Internet Protocol (IP) network. Directory services play an important role in developing intranet and Internet applications by allowing the sharing of information about users, systems, networks, services, and applications throughout the network. As examples, directory services may provide any organized set of records, often with a hierarchical structure, such as a corporate email directory. Similarly, a telephone directory is a list of subscribers with an address and a phone number. Port 389.

LDAPS is a distributed IP directory protocol similar to LDAP, but which incorporates SSL for greater security. Port 636.

Server Message Block (SMB), assoicated software referred to as SAMBA, is a protocol for sharing files, printers, serial ports, and communications abstractions such as named pipes and mail slots between computers. Port 445. Also,137,138,139

Internet Key Exchange (IKE) is a key management protocol standard used in conjunction with the Internet Protocol Security (IPSec) standard protocol. It provides security for virtual private networks’ (VPNs) negotiations and network access to random hosts. It can also be described as a method for exchanging keys for encryption and authentication over an unsecured medium, such as the Internet.

IKE enhances IPsec by providing additional features along with flexibility. IPsec, however, can be configured without IKE.

IKE has many benefits. It eliminates the need to manually specify all the IPSec security parameters at both peers. It allows the user to specify a particular lifetime for the IPsec security association. Furthermore, encryption can be changed during IPsec sessions. Moreover, it permits certification authority. Finally, it allows dynamic authentication of peers. Port 500.

Routing Information Protocol (RIP) is one of the oldest distance-vector routing protocols which employ the hop count as a routing metric. RIP prevents routing loops by implementing a limit on the number of hops allowed in a path from source to destination. The largest number of hops allowed for RIP is 15, which limits the size of networks that RIP can support. Port 520.

FTPS (also known as FTPES, FTP-SSL, and FTP Secure) is an extension to the commonly used File Transfer Protocol (FTP) that adds support for the Transport Layer Security (TLS) and, formerly, the Secure Sockets Layer (SSL, which is now prohibited by RFC7568) cryptographic protocols. Port 989/990.

FTPS should not be confused with the SSH File Transfer Protocol (SFTP), a secure file transfer subsystem for the Secure Shell (SSH) protocol with which it is not compatible. It is also different from FTP over SSH, which is the practice of tunneling FTP through an SSH connection.

SQL TCP port 1433 is the default port for SQL Server. This port is also the official Internet Assigned Number Authority (IANA) socket number for SQL Server. Client systems use TCP 1433 to connect to the database engine; SQL Server Management Studio (SSMS) uses the port to manage SQL Server instances across the network. You can reconfigure SQL Server to listen on a different port, but 1433 is by far the most common implementation. Port 1433.

Layer Two Forwarding (L2F) is a Cisco tunneling protocol that uses virtual dial-up networks for secure data packet transport. L2F functionality is similar to the Point-to-Point Tunneling Protocol (PPTP), which was developed by the Microsoft-led PPTP Forum. Port 1701.

Point-to-Point Tunneling Protocol (PPTP) Port 1723.

L2F creates point-to-point (PPP) network and user connections and allows high-level protocols to create tunnels via the link layer, including High-Level Data Link Control (HDLC) or SLIP frames. These tunnels separate server and termination points to facilitate network access.

In 1999, Microsoft and Cisco merged their respective L2F protocol versions and created L2TP, which is the Internet Engineering Task Force (IETF) tunneling standard.

H.323 is an ITU Telecommunication Standardization Sector (ITU-T) recommendation that describes protocols for the provision of audio-visual (A/V) communication sessions on all packet networks. H.323 provides standards for equipment, computers and services for multimedia communication across packet based networks and specifies transmission protocols for real-time video, audio and data details.

H.323 is widely used in IP based videoconferencing, Voice over Internet Protocol (VoIP) and Internet telephony. Users can communicate through the Internet and make use of a variety of products that are H.323 standard compatible. Port 1720.

RADIUS is a network protocol that is used to authenticate and authorize user access to a remote network. The term, RADIUS, is an acronym that stands for Remote Authentication Dial-In User Service. First introduced in 1991, RADIUS has remained a powerful tool for managing network user access. Port 1812,1813.

Media Gateway Control Protocol (MGCP) is a signaling and call control communications protocol used in voice over IP (VoIP) telecommunication systems. It implements the media gateway control protocol architecture for controlling media gateways connected to the public switched telephone network (PSTN). The media gateways provide conversion of traditional electronic media to the Internet Protocol (IP) network. The protocol is a successor to the Simple Gateway Control Protocol (SGCP), which was developed by Bellcore and Cisco, and the Internet Protocol Device Control (IPDC) Ports 2427, 2727.

Remote Desktop Protocol (RDP) is a proprietary protocol developed by Microsoft, which provides a user with a graphical interface to connect to another computer over a network connection. The user employs RDP client software for this purpose, while the other computer must run RDP server software.

Clients exist for most versions of Microsoft Windows (including Windows Mobile), Linux, Unix, macOS, iOS, Android, and other operating systems. RDP servers are built into Windows operating systems; an RDP server for Unix and OS X also exists. By default, the server listens on TCP port 3389 and UDP port 3389.

Real-time Transport Protocol (RTP) is a network protocol for delivering audio and video over IP networks. RTP is used in communication and entertainment systems that involve streaming media, such as telephony, video teleconference applications including WebRTC, television services and web-based push-to-talk features. Ports 5004, 5005.

The Session Initiation Protocol (SIP) is a signaling protocol used for initiating, maintaining, and terminating real-time sessions that include voice, video and messaging applications.[1] SIP is used for signaling and controlling multimedia communication sessions in applications of Internet telephony for voice and video calls, in private IP telephone systems, in instant messaging over Internet Protocol (IP) networks as well as mobile phone calling over LTE (VoLTE).

Ports:

5060 – SIP (unencrypted)
5061 – SIP (encrypted with TLS)

PuTTY is a free and open-source terminal emulator, serial console and network file transfer application. It supports several network protocols, including SCP, SSH, Telnet, rlogin, and raw socket connection. It can also connect to a serial port. The name “PuTTY” has no official meaning.

PuTTY was originally written for Microsoft Windows, but it has been ported to various other operating systems. Official ports are available for some Unix-like platforms, with work-in-progress ports to Classic Mac OS and macOS, and unofficial ports have been contributed to platforms such as Symbian,Windows Mobile and Windows Phone.

PuTTY was written and is maintained primarily by Simon Tatham, a British programmer.

Windows Server Update Services (WSUS) provides a cost-effective patch management solution to deploy updates to domain-joined Windows servers and workstation in a corporate network. WSUS is fully integrated in Windows Server 2012 and can be enabled on Windows clients by configuring settings in Group Policy Objects – GPO.

Secure Sockets Layer (SSL) is a standard protocol used for the secure transmission of documents over a network. Developed by Netscape, SSL technology creates a secure link between a Web server and browser to ensure private and integral data transmission. SSL uses Transport Control Protocol (TCP) for communication.

Internet Information Services (IIS) formerly known as Internet Information Server, is a web server producted by Microsoft. IIS is used with Microsoft Windows OSs and is the Microsoft-centric competition to Apache, the most popular webserver used with Unix/Linux-based systems.

Kiwi Syslog Server receives, logs, displays, alerts on, and forwards syslog, SNMP trap, and Windows event log messages from routers, switches, firewalls, Linux and UNIX hosts, and Windows machines.

Adaptive Security Device Manager (ASDM) is a Cisco technology that lets you manage Cisco Adaptive Security Appliance (ASA) firewalls and the Cisco AnyConnect Secure Mobility Client through a local, web-based interface.

AnyConnect is an Cisco Business App for Windows, iPhone and Android that facilitates user remote access to Cisco networks via VPN. (It is preferred method of remote access to Clientless SSL VPN)

Windows Remote Management (WinRM) is enabled in the firewall. WinRM is one component of the Windows Hardware Management features that manage server hardware locally and remotely.

AlienVault is a network monitoring system which uses OSSIM (Open Source Security Information Management) as a backbone of which it’s able to assist with intrusion detection and prevention operations.

Security Information and Event Management (SIEM) software products and services combine security information management (SIM) and security event management (SEM). They provide real-time analysis of security alerts generated by applications and network hardware.

Internet Control Message Protocol (ICMP) is a network protocol for Internet Protocol (IP) networking. ICMP transfers control information for the status of the network itself rather than application data. An IP network requires ICMP in order to function properly.

ICMP messages are a specific kind of IP message distinct from TCP and UDP.

The best-known example of ICMP messaging in practice is the ping utility, which uses ICMP to probe remote hosts for responsiveness and measure overall round-trip time of the probe messages. ICMP also supports other utilities like traceroute that identify intermediate routing devices (“hops”) on the path between a given source and destination.

pcap is an application programming interface (API) for capturing network traffic. While the name is an abbreviation of the technical term of art (jargon) packet capture, that is not the API’s proper name. Unix-like systems implement pcap in the libpcap library; for Windows, there is a port of libpcap named WinPcap that is no longer supported or developed, and a port named Npcap for Windows 7 and later that is still supported.

Monitoring software may use libpcap, WinPcap, or Npcap to capture network packets travelling over a computer network and, in newer versions, to transmit packets on a network at the link layer, and to get a list of network interfaces for possible use with libpcap, WinPcap, or Npcap.

Intrusion Detection Systems (IDS) analyze network traffic for signatures that match known cyberattacks.

Intrusion Prevention Systems (IPS) also analyzes packets, but can also stop the packet from being delivered based on what kind of attacks it detects — helping stop the attack.

Promiscuous Mode is a mode for a wired network interface controller (NIC) or wireless network interface controller (WNIC) that causes the controller to pass all traffic it receives to the central processing unit (CPU) rather than passing only the frames that the controller is specifically programmed to receive. This mode is normally used for packet sniffing that takes place on a router or on a computer connected to a wired network or one being part of a wireless LAN. Interfaces are placed into promiscuous mode by software bridges often used with hardware virtualization.

Universally Unique Identifier (UUID) is a 128-bit number used to identify information in computer systems. The term globally unique identifier (GUID) is also used, typically in software created by Microsoft.

OpenVAS is a framework of several services and tools offering a comprehensive and powerful vulnerability scanning and vulnerability management solution. The framework is part of Greenbone Networks’ commercial vulnerability management solution from which developments are contributed to the Open Source community since 2009.

The actual security scanner is accompanied with a regularly updated feed of Network Vulnerability Tests (NVTs), over 50,000 in total.

Damn Vulnerable Web App (DVWA) is a PHP/MySQL web application that is damn vulnerable. Its main goals are to be an aid for security professionals to test their skills and tools in a legal environment, help web developers better understand the processes of securing web applications and aid teachers/students to teach/learn web application security in a class room environment.

XAMPP is a free and open-source cross-platform web server solution stack package developed by Apache Friends, consisting mainly of the Apache HTTP Server, MariaDB database, and interpreters for scripts written in the PHP and Perl programming languages.

IPv4 and IPv6 are the versions of internet protocol where IPv6 is the enhanced version of IPv4. There are various differences between IPv4 and IPv6 protocol including their features, but the crucial one is the number of addresses (Address space) it generates.

IP version 4 (IPv4) generates 4.29 x 109 unique network addresses which is insufficient in quantity and as a result Internet is running out of space. Whereas IP version 6 (IPv6) produces 3.4 x 1038 addresses and is a scalable and flexible solution to the current problem.

Generic Routing Encapsulation (GRE) is a tunneling protocol developed by Cisco Systems that can encapsulate a wide variety of network layer protocols inside virtual point-to-point links or point-to-multipoint links over an Internet Protocol network. Both VPNs and GRE tunnels can be used to transfer data between remote locations. However, their similarities end there. If you are looking to provide a secure method of connecting remote users to resources stored within a central location, you should probably implement a VPN. However, if you need to pass traffic over an otherwise incompatible network, a GRE tunnel should be implemented. GRE tunnels don’t just have to run across the Internet; you can use them to join networks that couldn’t normally occur, perhaps enabling dynamic routing across a firewall for example. GRE tunnels can solve a few headaches in the real world so understanding them is very important.

One important point to note about GRE tunnels is that they do not inherently encrypt data. Everything that is sent across the tunnel is simply wrapped in GRE packets if you like. You will solve this issue in the next exercise by encrypting the data that is passed over the tunnel. This is especially important when creating a GRE tunnel over insecure networks such as the Internet.

Differences between VPN and GRE:

http://blog.boson.com/bid/92815/What-are-the-differences-between-an-IPSec-VPN-and-a-GRE-Tunnel

Multicast is group communication where data transmission is addressed to a group of destination computers simultaneously. Multicast can be one-to-many or many-to-many distribution. Multicast should not be confused with physical layer point-to-multipoint communication.

Group communication may either be application layer multicast[1] or network assisted multicast, where the latter makes it possible for the source to efficiently send to the group in a single transmission. Copies are automatically created in other network elements, such as routers, switches and cellular network base stations, but only to network segments that currently contain members of the group. Network assisted multicast may be implemented at the data link layer using one-to-many addressing and switching such as Ethernet multicast addressing, Asynchronous Transfer Mode (ATM), point-to-multipoint virtual circuits (P2MP)or Infiniband multicast. Network assisted multicast may also be implemented at the Internet layer using IP multicast. In IP multicast the implementation of the multicast concept occurs at the IP routing level, where routers create optimal distribution paths for datagrams sent to a multicast destination address.

Enhanced Interior Gateway Routing Protocol (EIGRP) is an advanced distance-vector routing protocol that is used on a computer network for automating routing decisions and configuration. The protocol was designed by Cisco Systems as a proprietary protocol, available only on Cisco routers. Functionality of EIGRP was converted to an open standard in 2013[1] and was published with informational status as RFC 7868 in 2016.

EIGRP is used on a router to share routes with other routers within the same autonomous system. Unlike other well known routing protocols, such as RIP, EIGRP only sends incremental updates, reducing the workload on the router and the amount of data that needs to be transmitted.

EIGRP replaced the Interior Gateway Routing Protocol (IGRP) in 1993. One of the major reasons for this was the change to classless IPv4 addresses in the Internet Protocol, which IGRP could not support.

Different types of NAT – Static NAT, Dynamic NAT and PAT

Static NAT (Network Address Translation) – Static NAT (Network Address Translation) is one-to-one mapping of a private IP address to a public IP address. Static NAT (Network Address Translation) is useful when a network device inside a private network needs to be accessible from internet.

Dynamic NAT (Network Address Translation) – Dynamic NAT can be defined as mapping of a private IP address to a public IP address from a group of public IP addresses called as NAT pool. Dynamic NAT establishes a one-to-one mapping between a private IP address to a public IP address. Here the public IP address is taken from the pool of IP addresses configured on the end NAT router. The public to private mapping may vary based on the available public IP address in NAT pool.

PAT (Port Address Translation) – Port Address Translation (PAT) is another type of dynamic NAT which can map multiple private IP addresses to a single public IP address by using a technology known as Port Address Translation.

Here when a client from inside network communicate to a host in the internet, the router changes the source port (TCP or UDP) number with another port number. These port mappings are kept in a table. When the router receive from internet, it will refer the table which keep the port mappings and forward the data packet to the original sender.

Errdisable (or Err–disable)is a feature that automatically disables a port on a Cisco Catalyst switch. When a port is error disabled, it is effectively shut down and no traffic is sent or received on that port.

The Errdisable error disable feature was designed to inform the administrator when there is a port problem or error. The reasons a catalyst switch can go into Errdisable mode and shutdown a port are many and include:

Duplex Mismatch
Loopback Error
Link Flapping (up/down)
Port Security Violation (psecure-violation)
Unicast Flodding
UDLD Failure
Broadcast Storms
BPDU Guard

Command Line Interface (CLI) Cheat Sheet: Networking

https://docs.paloaltonetworks.com/pan-os/7-1/pan-os-cli-quick-start/cli-cheat-sheets/cli-cheat-sheet-networking

VACLs (VLAN Access Control Lists) are another technology that you can use to help protect your network.

VACLs are particularly useful when you want to restrict access to certain hosts that may even be on the same VLAN. For example, you may have a scenario where you want to block communication between two servers on the same VLAN.

Denial-of-Service attack (DoS attack) is a cyber-attack in which the perpetrator seeks to make a machine or network resource unavailable to its intended users by temporarily or indefinitely disrupting services of a host connected to the Internet. Denial of service is typically accomplished by flooding the targeted machine or resource with superfluous requests in an attempt to overload systems and prevent some or all legitimate requests from being fulfilled.

Distributed denial-of-service (DDoS) attack is a malicious attempt to disrupt normal traffic of a targeted server, service or network by overwhelming the target or its surrounding infrastructure with a flood of Internet traffic. DDoS attacks achieve effectiveness by utilizing multiple compromised computer systems as sources of attack traffic. Exploited machines can include computers and other networked resources such as IoT devices. From a high level, a DDoS attack is like a traffic jam clogging up with highway, preventing regular traffic from arriving at its desired destination.

Personally identifiable information (PII) is information that, when used alone or with other relevant data, can identify an individual.

Cisco Internetwork Operating System (IOS) is a family of network operating systems used on many Cisco Systems routers and current Cisco network switches. Earlier, Cisco switches ran CatOS. IOS is a package of routing, switching, internetworking and telecommunications functions integrated into a multitasking operating system. Although the IOS code base includes a cooperative multitasking kernel, most IOS features have been ported to other kernels such as QNX and Linux for use in Cisco products.

RSA (Rivest–Shamir–Adleman), typically referred to as RSA key, is an algorithm used by modern computers to encrypt and decrypt messages. It is an asymmetric cryptographic algorithm. Asymmetric means that there are two different keys. This is also called public key cryptography, because one of the keys can be given to anyone. The other key must be kept private. The algorithm is based on the fact that finding the factors of a large composite number is difficult: when the factors are prime numbers, the problem is called prime factorization. It is also a key pair (public and private key) generator.

Out-of-band management is a device and system management technique primarily used in computer networking, but is also applied to other fields of IT where similar methods are used. This management method involves an alternate and dedicated connection to the system separate from the actual network that the system runs on. This allows an administrator to ensure the establishment of trust boundaries since there would only be a single entry point for the management interface.

Device management through out-of-band management is still done via a network connection, but this is entirely separate physically from the “in-band” network connection that the system is serving. Think of this as the door marked as “employees only” found in restaurants and shopping centers. No unauthorized user would be able to access the out-of-band network channel because there is simply no connection to it from the regular network channel that everyone uses, making this a very secure channel.

The out-of-band channel management interface is usually available even though the network may be down or even if the device is turned off, in sleep mode, hibernating or otherwise simply inaccessible through the operating system, making this a very powerful management control tool. This can be used to remotely manage powered-down devices outside working hours or during holidays, or it can be used to reboot devices whose operating system has hung or crashed. This is usually done through special operating system extensions and dedicated hardware specifically designed for out-of-band management.

An example configuration for out-of-band management is the use of Microsoft’s System Center together with Intel’s Active Management Technology and using Microsoft public key infrastructure.

Address Resolution Protocol (ARP) is used to determine the MAC Address of a node if you only know its IP address. When a host on a LAN wants to learn a MAC address, it sends out a broadcast ARP request: “Hey, anybody out there using 10.1.1.1?”

If there is, then the host will reply: “Yup, I’m 10.1.1.1 and I’m at 11:33:44:66:55:77!”

There is no relation between ARP and TCP. There is a direct correlation between ARP and IP, at the interface between layer 2 and layer 3.

The IP layer tells the ethernet layer that it needs a packet delivered to IP address 192.168.1.1. The ethernet layer checks its ARP table to see if the MAC address for that IP is known. If not, it broadcasts an ARP packet asking IP 192.168.1.1 to respond with its MAC address. The designated computer responds to the ARP request with its IP and MAC address. This info goes into the ARP table. At that point the IP packet is sent to the proper MAC address, and the receiving ethernet passes the packet up to the IP layer there. The packet may be TCP or UDP or any other protocol supported by IP.

Microsoft Baseline Security Analyzer (MBSA) The Microsoft Baseline Security Analyzer provides a streamlined method to identify missing security updates and common security misconfigurations.

Network Address Translation (NAT) is the process where a network device, usually a firewall, assigns a public address to a computer (or group of computers) inside a private network. The main use of NAT is to limit the number of public IP addresses an organization or company must use, for both economy and security purposes.

Access-Control List (ACL) with respect to a computer file system, is a list of permissions attached to an object. An ACL specifies which users or system processes are granted access to objects, as well as what operations are allowed on given objects.[1] Each entry in a typical ACL specifies a subject and an operation. For instance, if a file object has an ACL that contains (Alice: read,write; Bob: read), this would give Alice permission to read and write the file and Bob to only read it.

Open Web Application Security Project (OWASP) is an online community that produces freely-available articles, methodologies, documentation, tools, and technologies in the field of web application security.

OWASP ZAP (short for Zed Attack Proxy) is an open-source web application security scanner. It is intended to be used by both those new to application security as well as professional penetration testers. It is one of the most active Open Web Application Security Project (OWASP) projects and has been given Flagship status. When used as a proxy server it allows the user to manipulate all of the traffic that passes through it, including traffic using https.

National Institute of Standards and Technology (NIST) The NIST Cybersecurity Framework provides a policy framework of computer security guidance for how private sector organizations in the United States can assess and improve their ability to prevent, detect, and respond to cyber attacks.

Protected health information (PHI)

Whitelisting is the practice of explicitly allowing some identified entities access to a particular privilege, service, mobility, access or recognition. It is the reverse of blacklisting.

Federal Information Processing Standard Publication (FIPS)

Statement of Work (SOW)

Session hijacking, sometimes also known as cookie hijacking is the exploitation of a valid computer session—sometimes also called a session key—to gain unauthorized access to information or services in a computer system. In particular, it is used to refer to the theft of a magic cookie used to authenticate a user to a remote server. It has particular relevance to web developers, as the HTTP cookies[1] used to maintain a session on many web sites can be easily stolen by an attacker using an intermediary computer or with access to the saved cookies on the victim’s computer (see HTTP cookie theft). After successfully stealing appropriate session cookies an adversary might use the Pass the Cookie technique to perform session hijacking.

Zero-day (also known as 0-day) vulnerability is a computer-software vulnerability that is unknown to, or unaddressed by, those who should be interested in mitigating the vulnerability (including the vendor of the target software). Until the vulnerability is mitigated, hackers can exploit it to adversely affect computer programs, data, additional computers or a network.[1] An exploit directed at a zero-day is called a zero-day exploit, or zero-day attack.

SYN Short for synchronize, SYN is a TCP packet sent to another computer requesting that a connection be established between them. If the SYN is received by the second machine, an SYN/ACK is sent back to the address requested by the SYN. Lastly, if the original computer receives the SYN/ACK, a final ACK is sent.

SYN flood is a form of denial-of-service attack in which an attacker sends a succession of SYN requests to a target’s system in an attempt to consume enough server resources to make the system unresponsive to legitimate traffic.

A port scanner is an application designed to probe a server or host for open ports. Such an application may be used by administrators to verify security policies of their networks and by attackers to identify network services running on a host and exploit vulnerabilities.

A port scan or portscan is a process that sends client requests to a range of server port addresses on a host, with the goal of finding an active port; this is not a nefarious process in and of itself. The majority of uses of a port scan are not attacks, but rather simple probes to determine services available on a remote machine.

To portsweep is to scan multiple hosts for a specific listening port. The latter is typically used to search for a specific service, for example, an SQL-based computer worm may portsweep looking for hosts listening on TCP port 1433.

In cryptography, X.509 is a standard defining the format of public key certificates.[1] X.509 certificates are used in many Internet protocols, including TLS/SSL, which is the basis for HTTPS

In cryptography, Triple DES (3DES or TDES), officially the Triple Data Encryption Algorithm (TDEA or Triple DEA), is a symmetric-key block cipher, which applies the DES cipher algorithm three times to each data block. The Data Encryption Standard’s (DES) 56-bit key is no longer considered adequate in the face of modern cryptanalytic techniques and supercomputing power. However, an adapted version of DES, Triple DES (3DES), uses the same algorithm to produce a more secure encryption.

Public key cryptography standards (PKCS) are a group of specifications developed with the aim of accelerating the deployment of algorithms featuring two separate keys – one private and one public.

PKCS were first developed by RSA Laboratories with the cooperation of security developers from around the world.

The first published release of PKCS was in 1991 as a result of the cooperation of early adaptors. The standards promote the use of cryptography techniques such as the RSA algorithm and the Schnorr signature.

The Advanced Encryption Standard (AES), also known by its original name Rijndael (Dutch pronunciation: [ˈrɛindaːl]),[3] is a specification for the encryption of electronic data established by the U.S. National Institute of Standards and Technology (NIST) in 2001.[4]

AES is a subset of the Rijndael block cipher[3] developed by two Belgian cryptographers, Vincent Rijmen and Joan Daemen, who submitted a proposal[5] to NIST during the AES selection process.[6] Rijndael is a family of ciphers with different key and block sizes.

For AES, NIST selected three members of the Rijndael family, each with a block size of 128 bits, but three different key lengths: 128, 192 and 256 bits.

AES has been adopted by the U.S. government and is now used worldwide. It supersedes the Data Encryption Standard (DES),[7] which was published in 1977. The algorithm described by AES is a symmetric-key algorithm, meaning the same key is used for both encrypting and decrypting the data.

In the United States, AES was announced by the NIST as U.S. FIPS PUB 197 (FIPS 197) on November 26, 2001.[4] This announcement followed a five-year standardization process in which fifteen competing designs were presented and evaluated, before the Rijndael cipher was selected as the most suitable (see Advanced Encryption Standard process for more details).

AES became effective as a federal government standard on May 26, 2002, after approval by the Secretary of Commerce. AES is included in the ISO/IEC 18033-3 standard. AES is available in many different encryption packages, and is the first (and only) publicly accessible cipher approved by the National Security Agency (NSA) for top secret information when used in an NSA approved cryptographic module (see Security of AES, below).

A botnet is a number of Internet-connected devices, each of which is running one or more bots. Botnets can be used to perform distributed denial-of-service attack (DDoS attack), steal data,[1] send spam, and allows the attacker to access the device and its connection. The owner can control the botnet using command and control (C&C) software.[2] The word “botnet” is a combination of the words “robot” and “network”. The term is usually used with a negative or malicious connotation.

A command-and-control [C&C] server is a computer controlled by an attacker or cybercriminal which is used to send commands to systems compromised by malware and receive stolen data from a target network.

Industrial control systems (ICS)

Supervisory control and data acquisition (SCADA) is a control system architecture comprising computers, networked data communications and graphical user interfaces(GUI) for high-level process supervisory management, while also comprising other peripheral devices like programmable logic controllers (PLC) and discrete proportional-integral-derivative (PID) controllers to interface with process plant or machinery. The use of SCADA has been considered also for management and operations of project-driven-process in construction.

A domain controller (DC) is a server that responds to security authentication requests within a Windows Server domain. It is a server on a Microsoft Windows or Windows NT network that is responsible for allowing host access to Windows domain resources.

A domain controller is the centerpiece of the Windows Active Directory service. It authenticates users, stores user account information and enforces security policy for a Windows domain.

Advanced persistent threat (APT) is a stealthy computer network threat actor, typically a nation state or state-sponsored group, which gains unauthorized access to a computer network and remains undetected for an extended period.[1][2] In recent times, the term may also refer to non-state sponsored groups conducting large-scale targeted intrusions for specific goals

Common Vulnerabilities and Exposures (CVE®) is a list of entries — each containing an identification number, a description, and at least one public reference — for publicly known cybersecurity vulnerabilities.

Microsoft System Center Configuration Manager (SCCM), formerly Systems Management Server is a systems management software product developed by Microsoft for managing large groups of computers running Windows NT, Windows Embedded, macOS, Linux or UNIX, as well as Windows Phone, Symbian, iOS and Android mobile operating systems.

Security Assertion Markup Language (SAML) is an XML-based framework for authentication and authorization between two entities: a Service Provider and an Identity Provider. The Service Provider agrees to trust the Identity Provider to authenticate users. In return, the Identity provider generates an authentication assertion, which indicates that a user has been authenticated.

How to fix the “Fatal Error: Illegal characters in path” problem during a WSUS Installation

March 2, 2020security, System, Windows AdministrationDrive, Fatal Error: Illegal characters in path, guide, How To, howto, installion, lesson, tutorial, WSUSTidbytez 1

If your reading this you’re probably at the point of trying to install a new WSUS server on Windows Server 2012 R2 and have received the message “Fatal Error: Illegal characters in path”.

The fix is annoyingly simple.

You may have created a partition specifically for the task, i.e. created a new drive and assigned it the letter E:\, and only referenced the root of the partition assuming the installation would create whatever directories it needs, you know like every other software.

The problem is you cannot specify the root of partition alone, you need to specify a directory, or the partition name, such as E:\WSUS.

A fully qualified path should fix the problem.

What is WSUS?

Windows Server Update Services (WSUS) provides a cost-effective patch management solution to deploy updates to domain-joined Windows servers and workstation in a corporate network. WSUS is fully integrated in Windows Server 2012 and can be enabled on Windows clients by configuring settings in Group Policy Objects – GPO.

How to create a job that will test whether SQL Server database mail is working

November 2, 2019Coding, Database Administrator, dba, Development, IT, SQL Server, System, T-SQL, tsqldatabase mail, guide, How To, howto, Job, lesson, sql agent, SQL Server, SQL Server test email, SQL Server test email is working, SQL Server test mail, sql sever agent, test, tutorial, workingTidbytez

The following script will create a job that will run every minute to test if database mail can be sent from a job scheduled to run by the Sql Server Agent.

Simply find and replace the email address below with the email address you want to target:

testoperator@mail.com

Then run the script.

The operator ‘Test Operator’ and job ‘MailTest’ will be created.

The job is disabled by default, enable it to begin testing.

When you are finished run the commented out section at the bottom of the script to remove the test operator and job.

If you have just setup database mail for the first time the SQL Server Agent will need to be restarted.

/*
FIND AND REPLACE

testoperator@mail.com

*/
USE msdb;
GO

EXEC dbo.sp_add_operator @name = N'Test Operator'
	,@enabled = 1
	,@email_address = N'testoperator@mail.com'
GO

USE [msdb]
GO

BEGIN TRANSACTION

DECLARE @ReturnCode INT

SELECT @ReturnCode = 0

/****** Object:  JobCategory [[Uncategorized (Local)]]    Script Date: 31/07/2019 11:35:43 ******/
IF NOT EXISTS (
		SELECT NAME
		FROM msdb.dbo.syscategories
		WHERE NAME = N'[Uncategorized (Local)]'
			AND category_class = 1
		)
BEGIN
	EXEC @ReturnCode = msdb.dbo.sp_add_category @class = N'JOB'
		,@type = N'LOCAL'
		,@name = N'[Uncategorized (Local)]'

	IF (
			@@ERROR <> 0
			OR @ReturnCode <> 0
			)
		GOTO QuitWithRollback
END

DECLARE @jobId BINARY (16)

EXEC @ReturnCode = msdb.dbo.sp_add_job @job_name = N'MailTest'
	,@enabled = 0
	,@notify_level_eventlog = 0
	,@notify_level_email = 3
	,@notify_level_netsend = 0
	,@notify_level_page = 0
	,@delete_level = 0
	,@description = N'No description available.'
	,@category_name = N'[Uncategorized (Local)]'
	,@owner_login_name = N'sa'
	,@notify_email_operator_name = N'Test Operator'
	,@job_id = @jobId OUTPUT

IF (
		@@ERROR <> 0
		OR @ReturnCode <> 0
		)
	GOTO QuitWithRollback

/****** Object:  Step [Step 1]    Script Date: 31/07/2019 11:35:44 ******/
EXEC @ReturnCode = msdb.dbo.sp_add_jobstep @job_id = @jobId
	,@step_name = N'Step 1'
	,@step_id = 1
	,@cmdexec_success_code = 0
	,@on_success_action = 1
	,@on_success_step_id = 0
	,@on_fail_action = 2
	,@on_fail_step_id = 0
	,@retry_attempts = 0
	,@retry_interval = 0
	,@os_run_priority = 0
	,@subsystem = N'TSQL'
	,@command = N'SELECT 1'
	,@database_name = N'master'
	,@flags = 0

IF (
		@@ERROR <> 0
		OR @ReturnCode <> 0
		)
	GOTO QuitWithRollback

EXEC @ReturnCode = msdb.dbo.sp_update_job @job_id = @jobId
	,@start_step_id = 1

IF (
		@@ERROR <> 0
		OR @ReturnCode <> 0
		)
	GOTO QuitWithRollback

EXEC @ReturnCode = msdb.dbo.sp_add_jobschedule @job_id = @jobId
	,@name = N'Job Schedule'
	,@enabled = 1
	,@freq_type = 4
	,@freq_interval = 1
	,@freq_subday_type = 4
	,@freq_subday_interval = 1
	,@freq_relative_interval = 0
	,@freq_recurrence_factor = 0
	,@active_start_date = 20190731
	,@active_end_date = 99991231
	,@active_start_time = 0
	,@active_end_time = 235959
	,@schedule_uid = N'f0741db6-488e-44da-8f5e-a3f0ed13835e'

IF (
		@@ERROR <> 0
		OR @ReturnCode <> 0
		)
	GOTO QuitWithRollback

EXEC @ReturnCode = msdb.dbo.sp_add_jobserver @job_id = @jobId
	,@server_name = N'(local)'

IF (
		@@ERROR <> 0
		OR @ReturnCode <> 0
		)
	GOTO QuitWithRollback

COMMIT TRANSACTION

GOTO EndSave

QuitWithRollback:

IF (@@TRANCOUNT > 0)
	ROLLBACK TRANSACTION

EndSave:
GO

/*
REMOVE OPERATOR AND JOB
*/
/*
USE msdb;
GO

EXEC sp_delete_operator @name = 'Test Operator';

EXEC sp_delete_job @job_name = N'MailTest';

GO
*/

How to get SQL Server Network Information using SSMS

September 2, 2019Database Administrator, dba, IT, sql, SQL Server, System architecture, T-SQL, tsqlGET, Get network info using sql, guide, How get SQL Server Network info, How To, how to get network info using sql, howto, ip, ip address, lesson, SQL network info, sql sever, tutorialTidbytez

The following code will work for a remote client request to SQL 2008 and newer.

Note: The local machine address (local_net_address) is that of the SQL Server while client_net_address is the address of the remote computer you have used to make the request.

SELECT @@SERVERNAME AS ServerName
	,CONNECTIONPROPERTY('net_transport') AS net_transport
	,CONNECTIONPROPERTY('protocol_type') AS protocol_type
	,CONNECTIONPROPERTY('auth_scheme') AS auth_scheme
	,CONNECTIONPROPERTY('local_net_address') AS local_net_address
	,CONNECTIONPROPERTY('local_tcp_port') AS local_tcp_port
	,CONNECTIONPROPERTY('client_net_address') AS client_net_address

How to determine why a T-SQL command is unreasonably slow

September 2, 2018Database Administrator, dba, How To, sql, SQL Server, System, T-SQL, tsqlassess, command, fix, guide, How To, howto, lesson, Query, slow, small table, tutorial, unreasonably slowTidbytez 1

If you’ve ever found yourself in the situation were a command executing against a small table is nowhere near instant there can be numerous reasons for this but the most common causes are locks and waits.

The first step in identifying the problem is to execute the script below in a new query window while the troublesome command is running.

/* Queries Not Running */
SELECT ROW_NUMBER() OVER (
		ORDER BY r.total_elapsed_time DESC
		) AS Rn
	,st.TEXT AS SqlText
	,r.*
FROM sys.dm_exec_requests r
CROSS APPLY sys.dm_exec_sql_text(sql_handle) AS st
WHERE r.status <> 'running';

/* Queries Running */
SELECT ROW_NUMBER() OVER (
		ORDER BY r.total_elapsed_time DESC
		) AS Rn
	,st.TEXT AS SqlText
	,r.*
FROM sys.dm_exec_requests r
CROSS APPLY sys.dm_exec_sql_text(sql_handle) AS st
WHERE r.status = 'running';

This script will return two lists of the currently active sessions along with the stats associated with their execution. The first list will contain all the active sessions that are not running. The second list will contain all the active sessions that are running and will likely not contain the troublesome query you’re dealing with.

Identify your session based on the SqlText field. Be sure you’ve identified the session correctly as you may decide you want to kill the process later and killing the wrong one could cause you a lot of trouble.

status : If the status is not running look to the other fields in the returned result set to help identify the problem. If the session is in the running result set but you are unhappy with the performance it is likely the T-SQL needs to be optimized to make it run faster. This is a very broad topic and there are tons of articles and guides on the internet dealing with it.
blocking_session_id : If another session is blocking yours from executing, e.g. it has locked a table your command needs to write to, then this field will include the Id of the session causing the table to be locked. You can use EXEC sp_who2 to assess if the underlying command/query is experiencing a problem. If you are familiar with the blocking session you may know that you are able to kill the session without incurring any negative consequences. You can use the following code snippet to kill the blocking session.
```
KILL blocking_session_id /*replace by the actual Id*/
```
NOTE: Before you kill anything if it’s a command that has been running for a very long time it will likely take at least the same amount of time to roll back and unlock the table. You might be better off waiting for the session to finish on its own.
wait_type : If no blocking session is available, then the query is waiting for something, e.g. server resources etc. More details about wait types can be found HERE
wait_time : This stat value is measured in milliseconds. Short wait times are fine, specially in PAGEIOLATCH wait types (access to physical files) but longer wait times indicate a more serious problem.
last_wait_type : Indicates if the last wait type was different. This is quite helpful in analyzing if the query was blocked for the same reason before.

How to disable all enabled user job schedules in SQL Server with T-Sql

April 26, 2017Database Administrator, dba, How To, sql, SQL Server, System, T-SQL, tsqlall, disable, enabled, guide, How To, howto, Job, lesson, schedules, SQL Server, tutorial, with t-sqlTidbytez 1

I’d really recommend not running the output of this script on a live environment! This is just a little something I whipped up to test on a development environment.

Running the below script will output the T-Sql required to disable all enabled job schedules on the SQL Server instance. Just copy the outputted text and paste into a new SSMS window and execute to disable the jobs.

SET NOCOUNT ON

DECLARE @Schedule TABLE (ScheduleId INT)
DECLARE @ScheduleId AS INT
DECLARE @Sql AS VARCHAR(MAX)

SET @ScheduleId = ''

INSERT INTO @Schedule (ScheduleId)
SELECT s.schedule_id
FROM msdb.dbo.sysschedules AS s
WHERE s.[enabled] = 1
	AND s.owner_sid <> 0x01
ORDER BY s.NAME ASC

WHILE @ScheduleId IS NOT NULL
BEGIN
	SET @ScheduleId = (
			SELECT MIN(ScheduleId)
			FROM @Schedule
			WHERE ScheduleId > @ScheduleId
			)
	SET @Sql = '
EXEC msdb.dbo.sp_update_schedule @schedule_id = ' + '''' + CONVERT(VARCHAR(5), @ScheduleId) + '''' + '
	,@enabled = 0; 
'

	PRINT @Sql
END

Share this:

Like this:

Introduction

Project Management Triangle Misuse

IT Project Failure: Management Level Causes

Poor Project Visibility

Inadequate Domain Knowledge

Lack of Accountability

Conclusion

Share this:

Like this:

Summary

Introduction

What is a Data Warehouse?

The Digital Universe

Where is all this data coming from?

What types of data are there?

What is a Data Lake?

Data Lakes Scalability

Data Lakes Analysis & Analytics

Conclusion

References:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this: