TensorFlow Cheatsheet

Losses and Metrics

Use this TensorFlow reference while you build software engineering projects, review code for technical interview prep, or polish examples for a software engineer resume.

Using Losses and Metrics

Losses and metrics share a similar API. Losses are minimized; metrics are just monitored.

from tensorflow.keras import losses, metrics

# Pass as string to compile
model.compile(loss='mse', metrics=['mae', 'accuracy'])

# Pass as class instance (allows custom args)
model.compile(
    loss=losses.CategoricalCrossentropy(label_smoothing=0.1),
    metrics=[metrics.Accuracy(), metrics.AUC(curve='PR')],
)

# Call directly (returns scalar tensor)
loss_fn = losses.MeanSquaredError()
loss_val = loss_fn(y_true, y_pred)

# Reduction modes
losses.MeanSquaredError(reduction='sum_over_batch_size')   # default
losses.MeanSquaredError(reduction='sum')
losses.MeanSquaredError(reduction='none')  # per-sample, then reduce manually

Regression Losses

Class / String	Formula	Notes
`MeanSquaredError` / `'mse'`	mean((y - ŷ)²)	Penalizes large errors heavily
`MeanAbsoluteError` / `'mae'`	mean(\|y - ŷ\|)	Robust to outliers
`MeanAbsolutePercentageError` / `'mape'`	100 * mean(\|y-ŷ\|/\|y\|)	Percentage scale
`MeanSquaredLogarithmicError` / `'msle'`	mean((log(y+1)-log(ŷ+1))²)	For log-scaled targets
`Huber`	L2 near 0, L1 far from 0	Balanced outlier robustness
`LogCosh`	log(cosh(ŷ - y))	Smoother Huber

losses.Huber(delta=1.0)        # L2 for |error| <= delta, L1 beyond
losses.LogCosh()
losses.MeanAbsoluteError()

Classification Losses

Class / String	Use when
`BinaryCrossentropy` / `'binary_crossentropy'`	Binary labels (0/1), sigmoid output
`CategoricalCrossentropy` / `'categorical_crossentropy'`	One-hot labels, softmax output
`SparseCategoricalCrossentropy` / `'sparse_categorical_crossentropy'`	Integer labels, softmax output
`BinaryFocalCrossentropy`	Imbalanced binary classification
`CategoricalFocalCrossentropy`	Imbalanced multi-class
`KLDivergence` / `'kl_divergence'`	Distribution matching (VAE)
`Poisson`	Poisson regression

# Binary (sigmoid output, labels in {0,1})
losses.BinaryCrossentropy(from_logits=False, label_smoothing=0.0)

# Use from_logits=True when output layer has no activation (numerically stable)
losses.BinaryCrossentropy(from_logits=True)

# Multi-class with one-hot labels (softmax output)
losses.CategoricalCrossentropy(label_smoothing=0.1)

# Multi-class with integer labels (softmax output) — most common
losses.SparseCategoricalCrossentropy(from_logits=False)

# Focal loss (down-weights easy examples)
losses.BinaryFocalCrossentropy(alpha=0.25, gamma=2.0)
losses.CategoricalFocalCrossentropy(alpha=0.25, gamma=2.0)

Ranking / Similarity Losses

losses.CosineSimilarity(axis=-1)   # negate to use as a loss (maximized = loss minimized)
losses.Hinge()                     # SVM-style: max(0, 1 - y_true * y_pred)
losses.SquaredHinge()
losses.CategoricalHinge()

# Contrastive (manual)
def contrastive_loss(y_true, y_pred, margin=1.0):
    sq   = tf.square(y_pred)
    mar  = tf.square(tf.maximum(margin - y_pred, 0))
    return tf.reduce_mean(y_true * sq + (1 - y_true) * mar)

Custom Loss Function

# Simple function
def my_loss(y_true, y_pred):
    return tf.reduce_mean(tf.abs(y_true - y_pred) ** 1.5)

model.compile(loss=my_loss)

# Class (supports get_config, serialization)
class WeightedMSE(keras.losses.Loss):
    def __init__(self, weight=1.0, **kwargs):
        super().__init__(**kwargs)
        self.weight = weight

    def call(self, y_true, y_pred):
        return self.weight * tf.reduce_mean(tf.square(y_true - y_pred))

    def get_config(self):
        config = super().get_config()
        config['weight'] = self.weight
        return config

Classification Metrics

Metric	Notes
`Accuracy`	fraction correct (thresholded at 0.5 for binary)
`BinaryAccuracy`	binary: `y_pred > threshold`
`CategoricalAccuracy`	one-hot labels
`SparseCategoricalAccuracy`	integer labels
`TopKCategoricalAccuracy(k=5)`	correct in top-k
`SparseTopKCategoricalAccuracy(k=5)`	integer labels
`AUC(curve='ROC')`	area under ROC curve
`AUC(curve='PR')`	area under precision-recall curve
`Precision`	TP / (TP + FP)
`Recall`	TP / (TP + FN)
`F1Score` (Keras 3)	harmonic mean of P and R
`TruePositives` / `TrueNegatives` etc.	confusion matrix cells
`FalsePositives` / `FalseNegatives`	confusion matrix cells
`PrecisionAtRecall(recall=0.9)`	precision at target recall
`RecallAtPrecision(precision=0.9)`
`SensitivityAtSpecificity`
`SpecificityAtSensitivity`

metrics.AUC(curve='ROC', num_thresholds=200, multi_label=False)
metrics.Precision(thresholds=0.5)
metrics.Recall(class_id=1)   # per-class in multi-label
metrics.F1Score(average='macro', threshold=0.5)   # Keras 3

Regression Metrics

Metric	Notes
`MeanSquaredError`	same as MSE loss
`RootMeanSquaredError`	√MSE
`MeanAbsoluteError`	MAE
`MeanAbsolutePercentageError`	MAPE
`MeanSquaredLogarithmicError`	MSLE
`CosineSimilarity`	cosine similarity score
`LogCoshError`	log-cosh error
`R2Score` (Keras 3)	coefficient of determination

Using Metrics Manually

m = metrics.MeanSquaredError()

for x_batch, y_batch in val_dataset:
    y_pred = model(x_batch, training=False)
    m.update_state(y_batch, y_pred)

print(m.result().numpy())
m.reset_state()   # call before each epoch

Custom Metric

class MeanPrediction(keras.metrics.Metric):
    def __init__(self, name='mean_pred', **kwargs):
        super().__init__(name=name, **kwargs)
        self.total = self.add_weight(name='total', initializer='zeros')
        self.count = self.add_weight(name='count', initializer='zeros')

    def update_state(self, y_true, y_pred, sample_weight=None):
        self.total.assign_add(tf.reduce_sum(y_pred))
        self.count.assign_add(tf.cast(tf.size(y_pred), tf.float32))

    def result(self):
        return self.total / self.count

    def reset_state(self):
        self.total.assign(0.0)
        self.count.assign(0.0)

Loss Weighting Tricks

# Label smoothing (built in)
losses.CategoricalCrossentropy(label_smoothing=0.1)

# Per-sample weights via sample_weight
model.fit(x, y, sample_weight=weights_array)

# Class weights for imbalanced data
model.fit(x, y, class_weight={0: 1.0, 1: 5.0})

# Focal loss hyperparameters
losses.BinaryFocalCrossentropy(
    apply_class_balancing=True,   # use alpha weighting
    alpha=0.25,
    gamma=2.0,
)