Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2911 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5169. Min Range: -3.5169

Max Act: 3.5131. Min Act: -0.1700

Data Index: 6816830 (Open Web Text)

Max Activating Token Index: 451

Text #1

Max Range: 3.5169. Min Range: -3.5169

Max Act: 3.5169. Min Act: -0.1700

Data Index: 2311334 (Open Web Text)

Max Activating Token Index: 299

Text #2

Max Range: 3.5169. Min Range: -3.5169

Max Act: 3.4960. Min Act: -0.1700

Data Index: 6808626 (Open Web Text)

Max Activating Token Index: 160

Text #3

Max Range: 3.5169. Min Range: -3.5169

Max Act: 3.4727. Min Act: -0.1700

Data Index: 8224858 (Open Web Text)

Max Activating Token Index: 1022

Text #4

Max Range: 3.5169. Min Range: -3.5169

Max Act: 3.4608. Min Act: -0.1700

Data Index: 7942271 (Open Web Text)

Max Activating Token Index: 380

Text #5

Max Range: 3.5169. Min Range: -3.5169

Max Act: 3.4621. Min Act: -0.1700

Data Index: 7780083 (Open Web Text)

Max Activating Token Index: 908

Text #6

Max Range: 3.5169. Min Range: -3.5169

Max Act: 3.4727. Min Act: -0.1700

Data Index: 8683698 (Open Web Text)

Max Activating Token Index: 237

Text #7

Max Range: 3.5169. Min Range: -3.5169

Max Act: 3.4455. Min Act: -0.1700

Data Index: 8661546 (Open Web Text)

Max Activating Token Index: 833

Text #8

Max Range: 3.5169. Min Range: -3.5169

Max Act: 3.4567. Min Act: -0.1700

Data Index: 1543154 (Open Web Text)

Max Activating Token Index: 44

Text #9

Max Range: 3.5169. Min Range: -3.5169

Max Act: 3.4392. Min Act: -0.1700

Data Index: 4010226 (Open Web Text)

Max Activating Token Index: 154

Text #10

Max Range: 3.5169. Min Range: -3.5169

Max Act: 3.4251. Min Act: -0.1700

Data Index: 6703287 (Open Web Text)

Max Activating Token Index: 901

Text #11

Max Range: 3.5169. Min Range: -3.5169

Max Act: 3.4219. Min Act: -0.1700

Data Index: 2720267 (Open Web Text)

Max Activating Token Index: 89

Text #12

Max Range: 3.5169. Min Range: -3.5169

Max Act: 3.3949. Min Act: -0.1700

Data Index: 4719835 (Open Web Text)

Max Activating Token Index: 403

Text #13

Max Range: 3.5169. Min Range: -3.5169

Max Act: 3.3928. Min Act: -0.1700

Data Index: 2168115 (Open Web Text)

Max Activating Token Index: 378

Text #14

Max Range: 3.5169. Min Range: -3.5169

Max Act: 3.3925. Min Act: -0.1700

Data Index: 1430274 (Open Web Text)

Max Activating Token Index: 95

Text #15

Max Range: 3.5169. Min Range: -3.5169

Max Act: 3.4090. Min Act: -0.1700

Data Index: 5941830 (Open Web Text)

Max Activating Token Index: 401

Text #16

Max Range: 3.5169. Min Range: -3.5169

Max Act: 3.3985. Min Act: -0.1700

Data Index: 7027650 (Open Web Text)

Max Activating Token Index: 421

Text #17

Max Range: 3.5169. Min Range: -3.5169

Max Act: 3.4035. Min Act: -0.1700

Data Index: 2320014 (Open Web Text)

Max Activating Token Index: 970

Text #18

Max Range: 3.5169. Min Range: -3.5169

Max Act: 3.3508. Min Act: -0.1700

Data Index: 1762019 (Open Web Text)

Max Activating Token Index: 481

Text #19

Max Range: 3.5169. Min Range: -3.5169

Max Act: 3.3668. Min Act: -0.1700

Data Index: 1060910 (Open Web Text)

Max Activating Token Index: 17