Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 737 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5255. Min Range: -1.5255

Max Act: 1.5240. Min Act: -0.1700

Data Index: 6160732 (Open Web Text)

Max Activating Token Index: 39

Text #1

Max Range: 1.5255. Min Range: -1.5255

Max Act: 1.5196. Min Act: -0.1700

Data Index: 399744 (Open Web Text)

Max Activating Token Index: 44

Text #2

Max Range: 1.5255. Min Range: -1.5255

Max Act: 1.5255. Min Act: -0.1700

Data Index: 5320042 (Open Web Text)

Max Activating Token Index: 51

Text #3

Max Range: 1.5255. Min Range: -1.5255

Max Act: 1.4996. Min Act: -0.1700

Data Index: 7272649 (Open Web Text)

Max Activating Token Index: 47

Text #4

Max Range: 1.5255. Min Range: -1.5255

Max Act: 1.5122. Min Act: -0.1700

Data Index: 5316422 (Open Web Text)

Max Activating Token Index: 46

Text #5

Max Range: 1.5255. Min Range: -1.5255

Max Act: 1.5042. Min Act: -0.1700

Data Index: 2055425 (Open Web Text)

Max Activating Token Index: 47

Text #6

Max Range: 1.5255. Min Range: -1.5255

Max Act: 1.4863. Min Act: -0.1700

Data Index: 3562331 (Open Web Text)

Max Activating Token Index: 46

Text #7

Max Range: 1.5255. Min Range: -1.5255

Max Act: 1.4968. Min Act: -0.1700

Data Index: 1689711 (Open Web Text)

Max Activating Token Index: 47

Text #8

Max Range: 1.5255. Min Range: -1.5255

Max Act: 1.4726. Min Act: -0.1700

Data Index: 5878673 (Open Web Text)

Max Activating Token Index: 40

Text #9

Max Range: 1.5255. Min Range: -1.5255

Max Act: 1.4826. Min Act: -0.1700

Data Index: 8311417 (Open Web Text)

Max Activating Token Index: 36

Text #10

Max Range: 1.5255. Min Range: -1.5255

Max Act: 1.4842. Min Act: -0.1700

Data Index: 7518650 (Open Web Text)

Max Activating Token Index: 48

Text #11

Max Range: 1.5255. Min Range: -1.5255

Max Act: 1.4842. Min Act: -0.1700

Data Index: 1022742 (Open Web Text)

Max Activating Token Index: 38

Text #12

Max Range: 1.5255. Min Range: -1.5255

Max Act: 1.4802. Min Act: -0.1700

Data Index: 1201869 (Open Web Text)

Max Activating Token Index: 56

Text #13

Max Range: 1.5255. Min Range: -1.5255

Max Act: 1.4851. Min Act: -0.1700

Data Index: 6899485 (Open Web Text)

Max Activating Token Index: 48

Text #14

Max Range: 1.5255. Min Range: -1.5255

Max Act: 1.4777. Min Act: -0.1700

Data Index: 1184602 (Open Web Text)

Max Activating Token Index: 52

Text #15

Max Range: 1.5255. Min Range: -1.5255

Max Act: 1.4750. Min Act: -0.1700

Data Index: 1079572 (Open Web Text)

Max Activating Token Index: 56

Text #16

Max Range: 1.5255. Min Range: -1.5255

Max Act: 1.4723. Min Act: -0.1700

Data Index: 3660084 (Open Web Text)

Max Activating Token Index: 45

Text #17

Max Range: 1.5255. Min Range: -1.5255

Max Act: 1.4790. Min Act: -0.1700

Data Index: 6176668 (Open Web Text)

Max Activating Token Index: 44

Text #18

Max Range: 1.5255. Min Range: -1.5255

Max Act: 1.4857. Min Act: -0.1700

Data Index: 3499867 (Open Web Text)

Max Activating Token Index: 46

Text #19

Max Range: 1.5255. Min Range: -1.5255

Max Act: 1.4681. Min Act: -0.1700

Data Index: 5799175 (Open Web Text)

Max Activating Token Index: 40