Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1867 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.6830. Min Range: -2.6830

Max Act: 2.6830. Min Act: -0.1700

Data Index: 2245777 (Open Web Text)

Max Activating Token Index: 321

Text #1

Max Range: 2.6830. Min Range: -2.6830

Max Act: 2.5496. Min Act: -0.1700

Data Index: 1044852 (Open Web Text)

Max Activating Token Index: 179

Text #2

Max Range: 2.6830. Min Range: -2.6830

Max Act: 2.5043. Min Act: -0.1700

Data Index: 2018170 (Open Web Text)

Max Activating Token Index: 465

Text #3

Max Range: 2.6830. Min Range: -2.6830

Max Act: 2.5089. Min Act: -0.1700

Data Index: 2936841 (Open Web Text)

Max Activating Token Index: 258

Text #4

Max Range: 2.6830. Min Range: -2.6830

Max Act: 2.4981. Min Act: -0.1700

Data Index: 5620401 (Open Web Text)

Max Activating Token Index: 1006

Text #5

Max Range: 2.6830. Min Range: -2.6830

Max Act: 2.4922. Min Act: -0.1700

Data Index: 4148237 (Open Web Text)

Max Activating Token Index: 498

Text #6

Max Range: 2.6830. Min Range: -2.6830

Max Act: 2.5463. Min Act: -0.1700

Data Index: 2555627 (Open Web Text)

Max Activating Token Index: 189

Text #7

Max Range: 2.6830. Min Range: -2.6830

Max Act: 2.5390. Min Act: -0.1700

Data Index: 2535231 (Open Web Text)

Max Activating Token Index: 412

Text #8

Max Range: 2.6830. Min Range: -2.6830

Max Act: 2.4772. Min Act: -0.1700

Data Index: 284008 (Open Web Text)

Max Activating Token Index: 508

Text #9

Max Range: 2.6830. Min Range: -2.6830

Max Act: 2.4871. Min Act: -0.1700

Data Index: 7678360 (Open Web Text)

Max Activating Token Index: 839

Text #10

Max Range: 2.6830. Min Range: -2.6830

Max Act: 2.4213. Min Act: -0.1700

Data Index: 4626366 (Open Web Text)

Max Activating Token Index: 582

Text #11

Max Range: 2.6830. Min Range: -2.6830

Max Act: 2.3981. Min Act: -0.1700

Data Index: 5475458 (Open Web Text)

Max Activating Token Index: 263

Text #12

Max Range: 2.6830. Min Range: -2.6830

Max Act: 2.3929. Min Act: -0.1700

Data Index: 6697714 (Open Web Text)

Max Activating Token Index: 250

Text #13

Max Range: 2.6830. Min Range: -2.6830

Max Act: 2.3895. Min Act: -0.1700

Data Index: 2028003 (Open Web Text)

Max Activating Token Index: 353

Text #14

Max Range: 2.6830. Min Range: -2.6830

Max Act: 2.4101. Min Act: -0.1700

Data Index: 5493827 (Open Web Text)

Max Activating Token Index: 339

Text #15

Max Range: 2.6830. Min Range: -2.6830

Max Act: 2.3956. Min Act: -0.1700

Data Index: 8400222 (Open Web Text)

Max Activating Token Index: 353

Text #16

Max Range: 2.6830. Min Range: -2.6830

Max Act: 2.3778. Min Act: -0.1700

Data Index: 4119311 (Open Web Text)

Max Activating Token Index: 320

Text #17

Max Range: 2.6830. Min Range: -2.6830

Max Act: 2.4331. Min Act: -0.1700

Data Index: 7710294 (Open Web Text)

Max Activating Token Index: 541

Text #18

Max Range: 2.6830. Min Range: -2.6830

Max Act: 2.3793. Min Act: -0.1700

Data Index: 7378109 (Open Web Text)

Max Activating Token Index: 998

Text #19

Max Range: 2.6830. Min Range: -2.6830

Max Act: 2.3966. Min Act: -0.1700

Data Index: 2160535 (Open Web Text)

Max Activating Token Index: 99