Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 743 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0036. Min Range: -4.0036

Max Act: 4.0036. Min Act: -0.1700

Data Index: 4690959 (Open Web Text)

Max Activating Token Index: 378

Text #1

Max Range: 4.0036. Min Range: -4.0036

Max Act: 3.8562. Min Act: -0.1700

Data Index: 7531520 (Open Web Text)

Max Activating Token Index: 474

Text #2

Max Range: 4.0036. Min Range: -4.0036

Max Act: 3.8579. Min Act: -0.1700

Data Index: 4872421 (Open Web Text)

Max Activating Token Index: 482

Text #3

Max Range: 4.0036. Min Range: -4.0036

Max Act: 3.8455. Min Act: -0.1700

Data Index: 8531053 (Open Web Text)

Max Activating Token Index: 308

Text #4

Max Range: 4.0036. Min Range: -4.0036

Max Act: 3.7998. Min Act: -0.1700

Data Index: 3585549 (Open Web Text)

Max Activating Token Index: 34

Text #5

Max Range: 4.0036. Min Range: -4.0036

Max Act: 3.7975. Min Act: -0.1700

Data Index: 3380536 (Open Web Text)

Max Activating Token Index: 522

Text #6

Max Range: 4.0036. Min Range: -4.0036

Max Act: 3.7847. Min Act: -0.1700

Data Index: 6746142 (Open Web Text)

Max Activating Token Index: 34

Text #7

Max Range: 4.0036. Min Range: -4.0036

Max Act: 3.7465. Min Act: -0.1700

Data Index: 5599646 (Open Web Text)

Max Activating Token Index: 860

Text #8

Max Range: 4.0036. Min Range: -4.0036

Max Act: 3.7501. Min Act: -0.1700

Data Index: 3992620 (Open Web Text)

Max Activating Token Index: 92

Text #9

Max Range: 4.0036. Min Range: -4.0036

Max Act: 3.7110. Min Act: -0.1700

Data Index: 7227388 (Open Web Text)

Max Activating Token Index: 551

Text #10

Max Range: 4.0036. Min Range: -4.0036

Max Act: 3.6893. Min Act: -0.1700

Data Index: 8785135 (Open Web Text)

Max Activating Token Index: 725

Text #11

Max Range: 4.0036. Min Range: -4.0036

Max Act: 3.7208. Min Act: -0.1700

Data Index: 4469777 (Open Web Text)

Max Activating Token Index: 193

Text #12

Max Range: 4.0036. Min Range: -4.0036

Max Act: 3.6892. Min Act: -0.1700

Data Index: 261885 (Open Web Text)

Max Activating Token Index: 426

Text #13

Max Range: 4.0036. Min Range: -4.0036

Max Act: 3.6946. Min Act: -0.1700

Data Index: 7811066 (Open Web Text)

Max Activating Token Index: 707

Text #14

Max Range: 4.0036. Min Range: -4.0036

Max Act: 3.7104. Min Act: -0.1700

Data Index: 5336854 (Open Web Text)

Max Activating Token Index: 421

Text #15

Max Range: 4.0036. Min Range: -4.0036

Max Act: 3.7174. Min Act: -0.1700

Data Index: 3795420 (Open Web Text)

Max Activating Token Index: 279

Text #16

Max Range: 4.0036. Min Range: -4.0036

Max Act: 3.7008. Min Act: -0.1700

Data Index: 7578825 (Open Web Text)

Max Activating Token Index: 752

Text #17

Max Range: 4.0036. Min Range: -4.0036

Max Act: 3.6881. Min Act: -0.1700

Data Index: 6705088 (Open Web Text)

Max Activating Token Index: 303

Text #18

Max Range: 4.0036. Min Range: -4.0036

Max Act: 3.6688. Min Act: -0.1700

Data Index: 2942755 (Open Web Text)

Max Activating Token Index: 914

Text #19

Max Range: 4.0036. Min Range: -4.0036

Max Act: 3.6669. Min Act: -0.1700

Data Index: 4205801 (Open Web Text)

Max Activating Token Index: 645